Petals Health Monitor

Bootstrap peers: ●●

Swarm: StableBeluga2 • falcon-180B-chat • Llama-2-70b-chat-hf • Llama-2-70b-hf • StableBeluga-13B • bloom-560m

Top contributors: nora (140 blocks) • 🦆FYY🌞 (80 blocks) • donutmaster (78 blocks) • 🐝jbsheds🐝 (12 blocks)

Model petals-team/StableBeluga2 (healthy):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...Ge4fZU	🦆FYY🌞	2.3.0.dev1	132 tok/s	194	2710	3848	bf16 (nf4)		4096	Direct	Show Hide 1.4 ms 132.7 ms 406.7 ms 43.4 ms 36.8 ms 42.5 ms 263.0 ms 58.1 ms 67.7 ms	40:80																																									●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...YVgPCf	🦆FYY🌞	2.3.0.dev1	132 tok/s	194	2710	3848	bf16 (nf4)		3424	Direct	Show Hide 257.0 ms 529.4 ms 142.4 ms 347.2 ms 42.4 ms 36.4 ms	0:40	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...Maciiv	donutmaster	2.2.0	251 tok/s	342	3384	5231	bf16 (nf4)		32768	Direct	Show Hide 448.0 ms 1.8 ms 653.4 ms 524.2 ms 320.4 ms	52:78																																																					●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...iHH9VX		2.3.0.dev1	0 tok/s	0	1	136	bf16		16384	Direct	Show Hide 79.3 ms 334.0 ms 77.6 ms	75:80																																																																												●	●	●	●	●
...Ms7MQc		2.2.0	142 tok/s	330	3402	709	bf16 (nf4)		32768	Relay	Show Hide 252.9 ms 24.7 ms 353.1 ms	0:26	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖
...zowkex		2.2.0.post1	220 tok/s	50	659	2394	bf16 (nf4)		16384	Direct	Show Hide 48.2 ms 164.9 ms 281.7 ms 2.2 ms	35:40																																				●	●	●	●	●
...EDDqbC		2.2.0.post1	220 tok/s	50	659	2394	bf16 (nf4)		16384	Direct	Show Hide 44.7 ms 174.8 ms 460.0 ms 3.5 ms	35:40																																				●	●	●	●	●
...MtcC9K	donutmaster	2.2.0	251 tok/s	342	3384	5231	bf16 (nf4)		32768	Direct	Show Hide 265.0 ms 14.2 ms 1.5 ms 274.3 ms 289.8 ms 255.4 ms 287.9 ms	26:52																											●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...khXquK	🐝jbsheds🐝	2.2.0	172 tok/s	465	13375	172	bf16 (nf4)		32768	Direct	Show Hide 185.3 ms 70.0 ms 321.4 ms 79.5 ms	68:80																																																																					●	●	●	●	●	●	●	●	●	●	●	●
...cJCDFh	donutmaster	2.2.0	251 tok/s	342	3384	5231	bf16 (nf4)		32768	Direct	Show Hide 304.3 ms 16.8 ms 235.3 ms	0:26	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...iZDo5B		2.2.0.post1	16 tok/s	2	2587	79	f16		16384	Relay	Show Hide 54.1 ms 283.6 ms 58.9 ms 52.1 ms 50.4 ms	40:43																																									●	●	●
...V7nCc2	nora	2.3.0.dev1	112 tok/s	336	4530	695	bf16 (nf4)		11776	Direct	Show Hide 148.7 ms 213.3 ms 245.8 ms 1206.4 ms 189.7 ms 277.7 ms 141.9 ms 164.6 ms 234.9 ms 178.7 ms 172.5 ms	0:80	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model tiiuae/falcon-180B-chat (broken):

This model is supposed to be used by our chatbot app only. We do not provide a public API for this model due to license restrictions.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model meta-llama/Llama-2-70b-chat-hf (broken):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...yRxpNN	nora	2.3.0.dev1	250 tok/s	282	4341	250	f16 (nf4)		12288	Direct	Show Hide 1.9 ms	40:60																																									●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...3WDEwC	nora	2.3.0.dev1	250 tok/s	282	4341	250	f16 (nf4)		12288	Direct	Show Hide 1.8 ms	20:40																					●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...hgzDfR	nora	2.3.0.dev1	250 tok/s	282	4341	250	f16 (nf4)		12288	Direct	Show Hide	0:20	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Not enough servers? Connect your GPU and increase Petals capacity!

Model meta-llama/Llama-2-70b-hf (broken):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model stabilityai/StableBeluga-13B (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...MJG993		2.2.0	580 tok/s	548	11881	3664	f16 (nf4)		8192	Direct		0:40	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model bigscience/bloom-560m (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...o5dzpE		2.3.0.dev1	4 tok/s	41	55	7711	bf16		4096	Relay	Show Hide 105.4 ms	0:24	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖
...2VjoMz	src.eco	2.3.0.dev1	5 tok/s	7	65	5837	bf16		4096	Direct	Show Hide 102.6 ms	0:24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Legend:

Status		Description
●	online	Up and running
●	joining	Loading blocks, joining soon
✖	unreachable	Unreachable from the Internet, see "Reachability issues" below
_	offline	Disconnected a few minutes ago

Reachability issues:

Servers may be unreachable due to port forwarding/NAT/firewall issues. Normally, they should become reachable in a few minutes, once they join a libp2p circuit relay. If your server fails to do that, please ask for help in the #running-a-server channel of our Discord.

Peer ID	Error message
12D3KooWGsjE4JDYuqfsmB3CV3ZLdjjRzLmLzKsHvZNDNqMs7MQc	routing: not found
12D3KooWLWaNiHX7TWoGPvz3GzDU6bGfNncBquYo53uhNfo5dzpE	Failed to connect in 5 sec. Firewall may be blocking connections