⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
chat server
notification service
group service
message storage service
CDN
media service
Load balanceer
U1
U2
U3
WC
WC
WC
Group DB
user DB
message DB
message queue
blob
client
char server
database cassandra
create group chat
groupchat /chat: id, name, metadata
chatParticipents: chatId, participentId(GSI)(max 100 entry for 1 chatId)
client: userId, client id(max 5)
1000 connection
client1: ws1 client2: ws2 client2: ws3
fetch participats of the chat, send message to all connected/online participent connection
blob attatchments
attatchments: id, url
cleitn2
deliver message with presigned url for an attachment
download media
Message: id, content,creator, timestamp(Secondary index), chatId
how do we send message to offline users?
Inbox: recipientId, messageId
(create entry for all participents max 100)
chat server deletes partitipents entry if messsge is delivered( particiment connection was present with chat server) when some participatns were offline, when comes backonline, chat server fetch all message from inbox table and try to deliver all then delte entry once delivered
To support Billions of users connecting and sending messages simulteniously, we need 10Ks of Chat Connections servers
chat server
chat server
chat server
client sends https handshake to start TCP websocket connection. LB knows which chat server which has lesser connection and create new connection with that server
Problem with multiple chat server node? client 1 can be connected with server 1, and client 2 with server 2 how client 1 will send message to client 2(partitcipents of a chat) ??
Sol1: use user based topic, but kafka do not support Billions of topics Sol2: consistent has ring, know already perticuatur user will be always connection to perticular chat server, using zoopker Sol3: Redis pub sub(do not guarntee delivery as it delivers once, but we have inbox table containing data for retryal)
user connection/chat registry
zookeeper
get connection details
client
chat server
db
blob
redis pub/sub
LB
redis creates internal hashmap<string topic(userid), socket> redis do not guarntee deliver, but we have inbox details stored in persistence storeage, we can retry with that data
user1: ws1, server 1 user2: ws2, server2
when user connects to , redis fetch all the clients subscribed to that topic , redis will fetch all the subscriber, then fetched connection server details fetch it and sent it to server 2. server 2 sends it to user 2 using connection details saved at server 2
Message Routing
message service
client1
kafka
flink
store chat to participants list in cache
Hbase
partition on chat id
can send duplicate message
assign idempotentkey to avoid duplicate data
Message Delivery
process change data capture
LB
chat1: client 1 client 2 client 3
chat server contaning client 2 connection
chat server contaning client 3 connection
client 2
client 3
How load balancer decide which chat server to connect to find recipient connection details?
LB
CS1
CS2
CSn
consistent hasing
zookeeper
hashfun(userId) :: returns serverId
new user
server 1
user 1
hashfun always sends user1 to server 1
1
2
handshake between server and client
occsisanal heartbeat to balance servers
change hash function response if there is change in numer of server