Can the Same Process Serve Both WebSocket + Http Rquest?

(why is this an important question to ask)

When building the chat app, we want to serve socket traffic to tell when a user is online and have the message be send as socket data package.

The application also have some other functionality such as login/logout. Those are typically served by http protocols. This is because the server has no need to push to client for these features.

We can separte these two functionalities. The web socket can talk to other HTTP services via libray such as Python’s Request.

At the API gateway level, these two types of services can be routed differently.

See this StackOverflow question for more details: https://serverfault.com/questions/575467/running-a-websocket-server-and-a-http-server-on-the-same-server

  1. achievable if two process running on to ip
  2. if two processes running on same ip but different ports
  3. if one process running on same ip and same port but can handle two protocols at the same time. For example flasksocket + flask can achieve this.

Should the Same Process Serve Both WebSocket and Http Traffic?

Depends on if we want to scale these two bakcend differently. If so, we should have SocketServer and Http Server be two separate backend services.

How Should the Http Traffic from the WebSockets be routed?

Should these triffic go through the API gateway again(as if they these internal traffics are from external)?

What is WebSocket?

WebSocket is a protocol allow bi-directional communication between the server and the client.

Server was not able to talk to client when relying on HTTP only.

how to implement websocket client.
how to implement websocket server.

See this link for moe details: https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy

Key problem: Which machine is a specific client connect to?

Solve this problem either with a message broker or with a session storage.

How Does Inter-Backends Communication Work?

We do not really need this for the chat app backend because wo have already introduced a message broker.

Each socket server can dynamically subscribe to topics(keyed by threadid). –> this turns out to be trickly because the gunicorn worker is ephemeral, when it does it brings the subscription thread away with its death.

We should use the presence service as the look up table to find who is online and who is offline. Once find out which host name to talk to from the presence service, we can issue http call directly to the host machine(which will have a flask server listening to a port and answer the HTTP request to send message to a specific connected socket).

What does the desired architecture look like? Why does this version need to have a presence service?

Load balancer connects clients to socket server. Round robin is enough. No need for sticky session. Because inter-backend server communication is facilitated by a message bus. Therefore we need a presence service(canbe hanled by the session state redis cluster). (Alternative architecture need to have each wbsocket server know about other’s exits and perform consistent hashing which no longer needs a dedicated message broker)

Using session to manager user’s auth state. The session state is stored in a redis cluster. The actual credentials are stored in a users table(relational db).

send message to users in a thread that is online:

  1. because online users connects to the web socket servers, and the servers subscribes to the message broker topics. This subscriber callback will go over the connected sockets and send the message via the socket connection. The message broker’s queue worker only need to post message to the recepiant queue(keyed by thread_id).

  2. send message to users in a thread that is offline The thread queue worker needs to look up all the members in the thread(full picture) and check the session service(sometimes called the presence server) to find out who in the threads are offline.
    For those people offline, this worker need to make an entry in the database. When user getsback online the next time, it will create a task to ask for past message from the thread.

  3. the presence service is actually a session storage in Redis

Evolution of Architecture

First phase, user login. Session Management, presence service, and thread(chat room) creation.

Second phase, the sender websocket server breaks down thread and use the presence server to findout hosts names and performa message delivery.

Third phase, decouple the sender websocket server. We do this because in the second phase, the performance of the sender web socket is the bottleneck. To fix this perf bottleneck, we will move work form the sender websocket server to dedicated worker groups(working off of a queue). More specifically, we are going to have workers process the message send request. When processing a mesasge, consult the presence table. Invoke http calls to the host machine the receipent connects to, waits for the ack back, handles the failed message redelivery.

How to debug the sessions in the redis?

From the redis-cli we can MONITOR the commands executed to the redis instance. One example log can be:

1717880997.096565 [0 172.18.0.6:43518] "GET" "session:b22caead-b761-4510-adda-aaf81b62ba28"
1717880997.104562 [0 172.18.0.6:43518] "SETEX" "session:b22caead-b761-4510-adda-aaf81b62ba28" "2678400" "\x80\x04\x95\x12\x00\x00\x00\x00\x00\x00\x00}\x94\x8c\n_permanent\x94\x88s."
1717881006.467428 [0 172.18.0.6:43518] "GET" "session:b22caead-b761-4510-adda-aaf81b62ba28"
1717881006.980050 [0 172.18.0.6:43518] "SETEX" "session:b22caead-b761-4510-adda-aaf81b62ba28" "2678400" "\x80\x04\x95\xd4\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\n_permanent\x94\x88\x8c\b_user_id\x94\x8c f358d1bbfa85471eab08f0519c69b1bb\x94\x8c\x06_fresh\x94\x88\x8c\x03_id\x94\x8c\x80f3b83a9ff7b60c0599837692d04b534449ec633441e03444ad419c1cbc39cd4f19864aee29c97c6b9375937b28767455fea1ddc5704c7fa0e44e9e08dca789e4\x94u."

Notice that the hex encoded string, that is pickle. We can use the folloiwng Python script to decode it:

➜  redis-debug git:(qqiu.presence) ✗ cat decode_pickle.py
import pickle
import argparse
import binascii

def main():
    # Set up argument parsing
    parser = argparse.ArgumentParser(description='Deserialize pickle data from command line.')
    parser.add_argument('pickle_data', type=str, help='The pickle serialized data string.')

    # Parse the arguments
    args = parser.parse_args()

    # Convert the hexadecimal string representation of the pickle data to bytes
    pickle_data = bytes(args.pickle_data, 'utf-8').decode('unicode_escape').encode('latin1')

    # Deserialize the pickle data
    deserialized_data = pickle.loads(pickle_data)

    # Print the deserialized data
    print(deserialized_data)

if __name__ == '__main__':
    main()

The important bit is the pickle_data = bytes(args.pickle_data, 'utf-8').decode('unicode_escape').encode('latin1') because taking strings from the argument needs to be:

  1. treated as bytes
  2. escape the ‘\x’ parts.