Lengthy explanation follows...

First, the scenario before this patch:

We have three clients 1,2,3 connected. The main thread loops through
them using rfbClientIteratorNext() (loop L1) and is currently at
client 2 i.e. client 2's cl_2->refCount is 1. At this point we need to
loop again through the clients, with cl_2->refCount == 1, i.e. do a
loop L2 nested within loop L1.

BUT: Now client 2 disconnects, it's clientInput thread terminates its
clientOutput thread and calls rfbClientConnectionGone(). This LOCKs
clientListMutex and WAITs for cl_2->refCount to become 0. This means
this thread waits for the main thread to release cl_2. Waiting, with
clientListMutex LOCKed!

Meanwhile, the main thread is about to begin the inner
rfbClientIteratorNext() loop L2. The first call to rfbClientIteratorNext()
LOCKs clientListMutex. BAAM. This mutex is locked by cl2's clientInput
thread and is only released when cl_2->refCount becomes 0. The main thread
would decrement cl_2->refCount when it would continue with loop L1. But
it's waiting for cl2's clientInput thread to release clientListMutex. Which
never happens since this one's waiting for the main thread to decrement
cl_2->refCount. DEADLOCK.

Now, situation with this patch:

Same as above, but when client 2 disconnects it's clientInput thread
rfbClientConnectionGone(). This again LOCKs clientListMutex, removes cl_2
from the linked list and UNLOCKS clientListMutex. The WAIT for
cl_2->refCount to become 0 is _after_ that. Waiting, with
clientListMutex UNLOCKed!

Therefore, the main thread can continue, do the inner loop L2 (now only
looping through 1,3 - 2 was removed from the linked list) and continue with
loop L1, finally decrementing cl_2->refCount, allowing cl2's clientInput
thread to continue and terminate. The resources held by cl2 are not free()'d
by rfbClientConnectionGone until cl2->refCount becomes 0, i.e. loop L1 has
released cl2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

I was debbuging some code tonight and i found a pointer that is not been
freed, so i think there is maybe a memory leak, so it is...

there is the malloc caller reverse order:

( malloc cl->statEncList )
	<- rfbStatLookupEncoding
	<- rfbStatRecordEncodingSent
	<- rfbSendCursorPos
	<- rfbSendFramebufferUpdate
	<- rfbProcessEvents

I didnt look the whole libvncserver api, but i am using
rfbReverseConnection with rfbProcessEvents, and then when the client
connection dies, i am calling a rfbShutdownServer and rfbScreenCleanup,
but the malloc at rfbStatLookupEncoding isnt been freed.

So to free the stats i added a rfbResetStats(cl) after rfbPrintStats(cl)
at rfbClientConnectionGone in rfbserver.c before free the cl pointer. (at
rfbserver.c line 555). And this, obviously, is correcting the memory leak.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

For backwards-compatibility reasons, we can only add struct members to the
end. That way, existing callers still can use newer libraries, as the
structs are always allocated by the library (and therefore guaranteed to
have the correct size) and still rely on the same position of the parts
the callers know about.

Reported by Luca Falavigna.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

I got annoyed having to specify -resizable all the time; I never use it in
another mode anymore, since I am on a netbook.

The option -no-resizable was added to be able to switch off that feature.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Also, make it warn more generally when no known encryption lib is
available.

Commented out the surrounding '#ifdef _LIBC' to build md5.o with
leading underscores. This is required to match the prototypes defined
in md5.h.

Conflicts, resolved manually:
	libvncserver/Makefile.am

gnutls_certificate_set_x509_trust_file() returns the number of processed
certs and _not_ GNUTLS_E_SUCCESS (0) on success!

Working with “silent make mode” makes debugging a lot of easier
since warnings wont shadowed by useless compiler noise

gnutls_certificate_set_x509_trust_file() returns the number of processed
certs and _not_ GNUTLS_E_SUCCESS (0) on success!

Conflicts:
	libvncserver/websockets.c

Do not consume the peeked data if no close frame was detected.

Add common/sha1.h and common/sha1.c so that we have the SHA routines
even if openssl is not available. From the IETF SHA RFC example code.

Remove the UTF-8 encoding hack. This was really just an experiment.

If the protocol passed in the handshake has "binary" then don't base64
encode for the HyBi protocol. This will allow noVNC to request the
binary data be passed raw and not base64 encoded. Unfortunately, the
client doesn't speak first in VNC protocol (bad original design). If
it did then we could determine whether to base64 encode or not based
on the first HyBi frame from the client and whether the binary bit is
set or not. Oh well.

Misc Cleanup:

- Always free response and buf in handshake routine.

- Remove some unused variables.

Move the hixie disconnect hack to websockets.c. Removed
the remaining websockets vars from rfbClientPtr, so all
websockets stuff is hidden behind an opaque pointer.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>