Petals Health Monitor

Bootstrap peers:

Swarm: StableBeluga2 Llama-2-70b-hf bloom bloom-560m

Top contributors:

Model petals-team/StableBeluga2 (healthy):

Server ID » Contributor ? Version Throughput » Precision ? LoRAs ? Cache ? Avl. ? Pings ? Served blocks
...2L6Q76 2.3.0.dev2 153 tok/s bf16 (nf4) 16384 Relay 0:21 _____________________
...3NNV6q 2.3.0.dev1 112 tok/s bf16 (nf4) 12288 Direct Show 0:80

Model meta-llama/Llama-2-70b-hf (broken):

Server ID » Contributor ? Version Throughput » Precision ? LoRAs ? Cache ? Avl. ? Pings ? Served blocks

Not enough servers? Connect your GPU and increase Petals capacity!

Model bigscience/bloom (broken):

This model is not officially supported. Use it at your own risk.

Server ID » Contributor ? Version Throughput » Precision ? LoRAs ? Cache ? Avl. ? Pings ? Served blocks
...PzwTM9 680WM 2.3.0.dev2 227 tok/s bf16 (nf4) 4096 Relay Show 0:10

Not enough servers? Connect your GPU and increase Petals capacity!

Model bigscience/bloom-560m (healthy):

This model is not officially supported. Use it at your own risk.

Server ID » Contributor ? Version Throughput » Precision ? LoRAs ? Cache ? Avl. ? Pings ? Served blocks
...YxGzR4 2.3.0.dev2 841 tok/s bf16 (nf4) 4096 Direct Show 0:24
...oouiPp src.eco 2.3.0.dev2 1 tok/s bf16 4096 Relay Show 0:24

Legend:

StatusDescription
onlineUp and running
joiningLoading blocks, joining soon
unreachableUnreachable from the Internet, see "Reachability issues" below
_offlineDisconnected a few minutes ago

Reachability issues:

Servers may be unreachable due to port forwarding/NAT/firewall issues. Normally, they should become reachable in a few minutes, once they join a libp2p circuit relay. If your server fails to do that, please ask for help in the #running-a-server channel of our Discord.

Peer IDError message
12D3KooWGh44t6uGQ3UvhRQ3bN6A3yUXBZganRJ8DjsX3BYxGzR4 routing: not found