Petals Health Monitor

Bootstrap peers: ●●

Swarm: StableBeluga2 • Llama-2-70b-hf • Meta-Llama-3-70B • bloom

Top contributors:

Model petals-team/StableBeluga2 (healthy):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...nuEbMK		2.3.0.dev2	30 tok/s	571	35236	150	bf16 (nf4)		16384	Relay	Show Hide 49.5 ms 202.6 ms	0:80	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...bAub5T		2.3.0.dev2	891 tok/s	419	18424	4454	bf16 (nf4)		16384	Relay	Show Hide 55.0 ms 188.6 ms	0:30	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...3NNV6q		2.3.0.dev1	112 tok/s	336	4530	695	bf16 (nf4)		16384	Direct	Show Hide 375.3 ms 318.4 ms	0:80	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model meta-llama/Llama-2-70b-hf (broken):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model meta-llama/Meta-Llama-3-70B (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...ijyUtz	🦆FYY🌞	2.3.0.dev2	132 tok/s	187	2703	2922	bf16 (nf4)		16384	Relay	Show Hide 1.5 ms 1.9 ms 136.5 ms	40:80																																									●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...6QBzTR	🦆FYY🌞	2.3.0.dev2	132 tok/s	187	2703	2922	bf16 (nf4)		16384	Relay	Show Hide 2.0 ms	0:40	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...DsTAuQ	PC-Memorelab	2.3.0.dev2	84 tok/s	744	24708	419	bf16 (nf4)		16384	Relay	Show Hide 136.3 ms	59:80																																																												●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...oRSJiD	🦆FYY🌞	2.3.0.dev2	821 tok/s	266	29929	4104	bf16 (nf4)		16384	Relay	Show Hide 1.8 ms	0:40	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model bigscience/bloom (broken):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...PzwTM9		2.3.0.dev2	227 tok/s	152	18326	1137	bf16 (nf4)		4096	Relay	Show Hide	0:10	✖	✖	✖	✖	✖	✖	✖	✖	✖	✖

Not enough servers? Connect your GPU and increase Petals capacity!

Legend:

Status		Description
●	online	Up and running
●	joining	Loading blocks, joining soon
✖	unreachable	Unreachable from the Internet, see "Reachability issues" below
_	offline	Disconnected a few minutes ago

Reachability issues:

Servers may be unreachable due to port forwarding/NAT/firewall issues. Normally, they should become reachable in a few minutes, once they join a libp2p circuit relay. If your server fails to do that, please ask for help in the #running-a-server channel of our Discord.

Peer ID	Error message
12D3KooWQkiQD9g4og6d9cfPsNqhDrFec3dyavjz7V1MG9PzwTM9	failed to find peers: routing: not found