Petals Health Monitor

Bootstrap peers: ●●

Swarm: Meta-Llama-3.1-405B-Instruct • Mixtral-8x22B-Instruct-v0.1 • Llama-2-70b-chat-hf • Reflection-Llama-3.1-70B • StableBeluga2 • Llama-2-13b-chat-hf • Llama-2-7b-hf • bloom-560m

Top contributors:

Model meta-llama/Meta-Llama-3.1-405B-Instruct (broken):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model mistralai/Mixtral-8x22B-Instruct-v0.1 (broken):

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model meta-llama/Llama-2-70b-chat-hf (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...fYWnnj	tx	2.3.0.dev2	718 tok/s	166	10397	3592	f16 (nf4)		16384	Relay	Show Hide > 5 s	53:80																																																						●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...5iv9Wx	bz	2.3.0.dev2	153 tok/s	166	9994	763	f16 (nf4)		16384	Relay	Show Hide	0:27	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...WDgPbE	kv	2.3.0.dev2	754 tok/s	157	10558	11000	f16 (nf4)		16384	Relay	Show Hide > 5 s	27:54																												●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model mattshumer/Reflection-Llama-3.1-70B (broken):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...codn5o		2.3.0.dev2	2 tok/s	97	3393	8	bf16 (nf4)		16384	Relay		0:5	●	●	●	●	●

Not enough servers? Connect your GPU and increase Petals capacity!

Model petals-team/StableBeluga2 (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...PUXqcs		2.3.0.dev2	205 tok/s	23	1742	5581	bf16		16384	Direct	Show Hide 19.9 ms	32:48																																	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●
...EWTFJu		2.3.0.dev2	112 tok/s	336	4530	695	bf16 (nf4)		15872	Direct	Show Hide 19.7 ms	0:80	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model meta-llama/Llama-2-13b-chat-hf (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...opy7n4	kt	2.3.0.dev2	900 tok/s	290	24880	4501	f16 (nf4)		4096	Relay		0:40	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model meta-llama/Llama-2-7b-hf (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...GQUxjj		2.3.0.dev2	2460 tok/s	1107	97068	12302	f16 (nf4)		4096	Relay		0:32	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Model bigscience/bloom-560m (healthy):

This model is not officially supported. Use it at your own risk.

Server ID »	Contributor ?	Version	Throughput »	Inference ?	Forward ?	Network ?	Precision ?	LoRAs ?	Cache ?	Avl. ?	Pings ?	Served blocks
...fkiYWT	src.eco	2.3.0.dev2	1 tok/s	13	18	6104	bf16		4096	Relay	Show Hide	0:24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●

Legend:

Status		Description
●	online	Up and running
●	joining	Loading blocks, joining soon
✖	unreachable	Unreachable from the Internet, see "Reachability issues" below
_	offline	Disconnected a few minutes ago