bufferevent: allow setting priority on socket and openssl type

(cherry picked from commit bdc5200acdf0a004ddc805b87bf09439fe2ebe7f)

Eliminate fd conversion warnings and introduce EVUTIL_INVALID_SOCKET (windows)

windows has intptr_t instead of regular int.

Also tt_fd_op() had been introduced, since we cannot use tt_int_op() for

Eliminate fd conversion warnings and introduce EVUTIL_INVALID_SOCKET (windows)

windows has intptr_t instead of regular int.

Also tt_fd_op() had been introduced, since we cannot use tt_int_op() for
comparing fd, since it is not always int.

(cherry picked from commit b29207dceee33832bb28ab103a833df6a2fd29d3)

show more ...

Use BEV_UPCASE() everywhere

Done with coccinelle and manual line rewrap:
$ cat > BEV_UPCAST.cocci
@@
expression field_;
expression var;
@@

- EVUTIL_UPCAST(var, struct bufferevent_privat

Use BEV_UPCASE() everywhere

Done with coccinelle and manual line rewrap:
$ cat > BEV_UPCAST.cocci
@@
expression field_;
expression var;
@@

- EVUTIL_UPCAST(var, struct bufferevent_private, field_)
+ BEV_UPCAST(var)

$ spatch --sp-file BEV_UPCASE.cocci --in-place bufferevent*.c > /dev/null

(cherry picked from commit 7bcf576b396b22f218074c446691526a3ce509a6)

show more ...

Fix evhttp_connection_get_addr() fox incomming http connections

Install conn_address of the bufferevent on incomping http connections
(even though this is kind of subsytem violation, so let's fix it

Fix evhttp_connection_get_addr() fox incomming http connections

Install conn_address of the bufferevent on incomping http connections
(even though this is kind of subsytem violation, so let's fix it in a
simplest way and thinkg about long-term solution).

Fixes: #510
Closes: #595 (pick)
(cherry picked from commit 367cd9e5c2b8e3f9b8dbed40bfe8a0ed79285c36)

show more ...

bufferevent_socket_connect{,_hostname}() missing event callback and use ret code

- When socket() failed in bufferevent_socket_connect() , the event
callback should be called also in
bufferevent_

bufferevent_socket_connect{,_hostname}() missing event callback and use ret code

- When socket() failed in bufferevent_socket_connect() , the event
callback should be called also in
bufferevent_socket_connect_hostname(). eg. when use
bufferevent_socket_connect_hostname() to resolve and connect an IP
address but process have a smaller ulimit open files, socket() fails
always but caller is not notified.

- Make bufferevent_socket_connect()'s behavior more consistent: function
return error then no callback, function return ok then error passed by
event callback.

Fixes: #597
Closes: #599
Closes: #600
(cherry picked from commit f7bc1337977553f5f966f13840a5f7feba2d0bd5)

show more ...

Remove check against passed bufferevent in bufferevent_socket_connect()

It is a bit confusing to check against the result of EVUTIL_UPCAST(),
because it can return not NULL for NULL pointers, but fo

Remove check against passed bufferevent in bufferevent_socket_connect()

It is a bit confusing to check against the result of EVUTIL_UPCAST(),
because it can return not NULL for NULL pointers, but for even though
with bufferevent we are fine (because bufferevent is the first field in
bufferevent_private), there are no checks for "bufev" in bufferevent's
API, so just remove it to make it generic.

Fixes: #542
Signed-off-by: Ivan Maidanski <[email protected]>
Signed-off-by: Azat Khuzhin <[email protected]>
(cherry picked from commit a10a6f4ed918ea1432820d99e9373f37f906d6f0)

show more ...

bufferevent: refactor to use type check macros

(cherry picked from commit 92cc0b9c3db38088f79c5d1e432c429fbc366968)

be: fix with filtered bufferevents and connect() without EAGAIN

With filtered bufferevents (i.e. not real one, that have socket), we can
trigger incorrect callback in this case. Let's look at exampl

be: fix with filtered bufferevents and connect() without EAGAIN

With filtered bufferevents (i.e. not real one, that have socket), we can
trigger incorrect callback in this case. Let's look at example with http
and bufferevent_openssl_filter_new():
- bev = bufferevent_openssl_filter_new()
- http layer trying to connect() to localhost with bev
# at this time, bev have writecb/readcb NULL but ev_write/ev_read has
# timeout with 45 secs, default HTTP connect timeout
- and when connect() retruns without EAGAIN (BSD'ism) we called
event_active() before (with EV_WRITE), and this will call ev_write
timeout only, while it is more correct to act on bufferevent instead
of plain event, so let's trigger EV_WRITE for bufferevent which will
do the job (and let's do this deferred).

Fixes: http/https_simple # under solaris

show more ...

be_sock: handle readv() returns ECONNREFUSED (freebsd 9.2)

During testing sometimes bufferevent/bufferevent_connect_fail_eventcb fails,
and after some digging I found that this was the case when con

be_sock: handle readv() returns ECONNREFUSED (freebsd 9.2)

During testing sometimes bufferevent/bufferevent_connect_fail_eventcb fails,
and after some digging I found that this was the case when connect() doesn't
return ECONNREFUSED immediately, and instead next operation will return it (in
our case evbuffer_read()->readv(), needless to say that after this
bufferevent_writecb() called and it checks error with getsockopt() but of
course it doesn't return any error), so this patch checks "errno" after
"readv()" and installs "bufev::connection_refused" flag, to handle this from
writecb and only once.

Fixes: bufferevent/bufferevent_connect_fail_eventcb
Fixes: bufferevent/bufferevent_connect_fail_eventcb_defer
Refs: #388

show more ...

be_sock: cancel in-progress dns requests

Before this patch we didn't have such functionality and this can cause some
issues when we are cancelling HTTP request while after this we will get a
respons

be_sock: cancel in-progress dns requests

Before this patch we didn't have such functionality and this can cause some
issues when we are cancelling HTTP request while after this we will get a
response/timeout from the NS and in this case error_cb will be called and can
damage newly started HTTP request.
We could also have problems with connect, but we don't have them since we
closes the fd in HTTP layer.

This is not so good that this is done in be_sock, but it is internal, so we can
change without pain. Plus I don't like that callback-via-evutil, but since we
have event_extra we need do like that.

And after this patch the following tests doesn't report leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
...
==10469== FILE DESCRIPTORS: 2309 open at exit.
...
==10469== HEAP SUMMARY:
==10469== in use at exit: 0 bytes in 0 blocks
==10469== total heap usage: 33,846 allocs, 33,846 frees, 4,617,651 bytes allocated
==10469==
==10469== All heap blocks were freed -- no leaks are possible

v2: do under lock
v3: reset dns request
v4: ignore EVUTIL_EAI_CANCEL in regular callback

show more ...

be_sock: evutil_getaddrinfo_async_() always return 0

be_sock: unfreeze buffers on fd changing

Only bufferevent_sock have evbuffer_freeze()/evbuffer_unfreeze() & ctrl ops, so
we don't need to fix other bufferevents (be_pair doesn't have ctrl op).

Foun

be_sock: unfreeze buffers on fd changing

Only bufferevent_sock have evbuffer_freeze()/evbuffer_unfreeze() & ctrl ops, so
we don't need to fix other bufferevents (be_pair doesn't have ctrl op).

Found during draining buffers in http layer, and hence 501-not-implemented
error in regress http/.. (with some custom hacking).

show more ...

be_sock: bufferevent_socket_connect_hostname(): make it thread-safe

If you use bufferevent_socket_connect_hostname() to resolve, then ipv4 answer
can be returned before ipv6 scheduled and if you wil

be_sock: bufferevent_socket_connect_hostname(): make it thread-safe

If you use bufferevent_socket_connect_hostname() to resolve, then ipv4 answer
can be returned before ipv6 scheduled and if you will destroy bufferevent after
ipv4 answer will come (in a separate thread of course) then ipv6 will trigger
UAF:

$ a.out
=================================================================
==29733==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200000ef50 at pc 0x0000004b7aef bp 0x7fffffffd940 sp 0x7fffffffd0f8
READ of size 2 at 0x60200000ef50 thread T0
#0 0x4b7aee in __interceptor_index (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4b7aee)
#1 0x5060eb in string_num_dots /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2739
#2 0x5078df in search_request_new /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:3214
#3 0x506afd in evdns_base_resolve_ipv6 /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2935
#4 0x50aa94 in evdns_getaddrinfo /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:4719
#5 0x51de4f in evutil_getaddrinfo_async_ /src/oss/libevent/libevent-github/.cmake-debug/../evutil.c:1567
#6 0x4fe023 in bufferevent_socket_connect_hostname /src/oss/libevent/libevent-github/.cmake-debug/../bufferevent_sock.c:519
#7 0x524f54 in evhttp_connection_connect_ /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2493
#8 0x525156 in evhttp_make_request /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2548
#9 0x52d373 in main (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d373)
#10 0x7ffff6849b44 in __libc_start_main /tmp/buildd/glibc-2.19/csu/libc-start.c:287
#11 0x445806 in _start (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x445806)

0x60200000ef50 is located 0 bytes inside of 15-byte region [0x60200000ef50,0x60200000ef5f)
freed by thread T1 here:
#0 0x4cc4f2 in __interceptor_free (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4cc4f2)
#1 0x5141c1 in event_mm_free_ /src/oss/libevent/libevent-github/.cmake-debug/../event.c:3512
#2 0x522402 in evhttp_connection_free /src/oss/libevent/libevent-github/.cmake-debug/../http.c:1206
#3 0x52cc5f in connection_closer (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52cc5f)
#4 0x50e80e in event_process_active_single_queue /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1642
#5 0x50ed57 in event_process_active /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1734
#6 0x50f458 in event_base_loop /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1957
#7 0x50eddf in event_base_dispatch /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1768
#8 0x52d075 in event_dispatch_thread (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d075)
#9 0x7ffff74fc0a3 in start_thread /tmp/buildd/glibc-2.19/nptl/pthread_create.c:309

Fixes: #217
Closes: #222
Closes: #219
Gist: https://gist.github.com/azat/92cbb34232ac02d7972b (from #217 but thread-safe)

show more ...

be_openssl: use bufferevent_enable() instead of bufferevent_add_event_()

By using bufferevent_enable() there will be no event for READ *or* WRITE if
they are not enabled before, and this patch reduc

be_openssl: use bufferevent_enable() instead of bufferevent_add_event_()

By using bufferevent_enable() there will be no event for READ *or* WRITE if
they are not enabled before, and this patch reduces difference for
be_sock_enable/be_openssl_enable (handshake)

show more ...

be_sock: drop be_sock_add() macro (useless and debug unfriendly)

be: introduce bufferevent_generic_adj_existing_timeouts_()

And use it in openssl/sock layers to avoid copy-pasting it's variants.

be_sock: bufferevent_socket_set_conn_address(): assert instead of silent no-op

be_sock: sanity check in bufferevent_socket_set_conn_address()

be: we don't need to use getpeername() we we have conn_address

Since getpeername() could fail.

be: replace conn_address by full struct instead of pointer

bufferevent: move conn_address out from http into bufferevent

In http the only case when when we could store it is when we already
connected, *but* if we are doing request using domain name, then we

bufferevent: move conn_address out from http into bufferevent

In http the only case when when we could store it is when we already
connected, *but* if we are doing request using domain name, then we need
to do request to nameserver to get IP address, and this is handled by
bufferevent.
So when we have IP address (from nameserver) and don't have connection
to this IP address, we could already cache it to avoid extra DNS
requests (since UDP is slow), and we can't do this from http layer, only
from bufferevent.

show more ...

be: make @sa const for bufferevent_socket_connect()

Add an option to trigger bufferevent event callbacks

Add an option to trigger bufferevent I/O callbacks

Use finalization feature so bufferevents can avoid deadlocks

Since the bufferevents' events are now EV_FINALIZE (name pending),
they won't deadlock. To clean up properly, though, we must use the
fi

Use finalization feature so bufferevents can avoid deadlocks

Since the bufferevents' events are now EV_FINALIZE (name pending),
they won't deadlock. To clean up properly, though, we must use the
finalization feature.

This patch also split bufferevent deallocation into an "unlink" step
that happens fast, and a "destruct" step that happens after
finalization.

More work is needed: there needs to be a way to specify a finalizer
for the bufferevent's argument itself. Also, this finalizer business
makes lots of the reference counting we were doing unnecessary.

Also, more testing is needed.

show more ...