Communication Protocols
Overview
Section titled “Overview”This page documents the network communication protocols and message formats used between system components in the NI Compute subnet. The system implements three core Bittensor synapse protocols (Specs
, Allocate
, Challenge
) and custom extensions to Bittensor’s communication infrastructure (ComputeSubnetAxon
, ComputeSubnetSubtensor
, ComputeSubnetAxonMiddleware
).
These protocols enable hardware specification discovery, resource allocation, and cryptographic validation challenges between validators and miners. For information about the HTTP API endpoints, see Resource Allocation API.
Sources: compute/protocol.py:1-136 , compute/axon.py:1-487
Protocol Stack
Section titled “Protocol Stack”NI Compute extends Bittensor’s communication infrastructure with custom protocols specific to the GPU compute marketplace. The system implements three main synapse protocols and uses customized versions of Bittensor’s Axon and Subtensor components.
Protocol Stack Architecture
graph TD subgraph "Bittensor Base Classes" bt_Synapse["bt.Synapse"] bittensor_Axon["bittensor.core.axon.Axon"] bittensor_Subtensor["bittensor.core.subtensor.Subtensor"] AxonMiddleware["bittensor.core.axon.AxonMiddleware"] end subgraph "compute/protocol.py" Specs["class Specs(bt.Synapse)"] --> bt_Synapse Allocate["class Allocate(bt.Synapse)"] --> bt_Synapse Challenge["class Challenge(bt.Synapse)"] --> bt_Synapse end subgraph "compute/axon.py" ComputeSubnetAxon["class ComputeSubnetAxon(axon)"] --> bittensor_Axon ComputeSubnetSubtensor["class ComputeSubnetSubtensor(subtensor)"] --> bittensor_Subtensor ComputeSubnetAxonMiddleware["class ComputeSubnetAxonMiddleware(AxonMiddleware)"] --> AxonMiddleware custom_serve_extrinsic["custom_serve_extrinsic()"] end subgraph "Communication Methods" preprocess["preprocess()"] --> ComputeSubnetAxonMiddleware serve_prometheus["serve_prometheus()"] --> ComputeSubnetSubtensor info["info()"] --> ComputeSubnetAxon end ComputeSubnetAxon --> ComputeSubnetAxonMiddleware
Sources: compute/protocol.py:23-136 , compute/axon.py:152-487
Core Synapse Protocols
Section titled “Core Synapse Protocols”NI Compute defines three primary protocols for communication between validators and miners, all extending Bittensor’s Synapse base class.
Specs Protocol
Section titled “Specs Protocol”The Specs
class enables validators to query hardware specifications from miners, providing detailed information about CPU, GPU, RAM, and storage resources.
Specs Class Definition
classDiagram class bt_Synapse { +deserialize() } class Specs { +str specs_input +str specs_output +deserialize() str } bt_Synapse <|-- Specs note for Specs "compute/protocol.py:23-57"
Attribute | Type | Description |
---|---|---|
specs_input | str | Input parameter (typically empty string) |
specs_output | str | JSON response containing detailed hardware specifications |
The deserialize()
method returns self.specs_output
containing hardware details in the format:
{"CPU":{'count' : 4, 'vendor_id_raw' : 'AuthenticAMD', ...}}
Sources: compute/protocol.py:23-57
Allocate Protocol
Section titled “Allocate Protocol”The Allocate
class facilitates resource allocation requests from validators to miners, including parameters for container configuration, access credentials, and resource requirements.
Allocate Class Definition
classDiagram class bt_Synapse { +deserialize() } class Allocate { +int timeline +dict device_requirement +bool checking +dict output +str public_key +dict docker_requirement +bool docker_change +dict docker_action +deserialize() dict } bt_Synapse <|-- Allocate note for Allocate "compute/protocol.py:60-110"
Attribute | Type | Default | Description |
---|---|---|---|
timeline | int | 0 | Duration of allocation in seconds |
device_requirement | dict | {} | Hardware requirements specification |
checking | bool | True | Flag: True for availability check, False for actual allocation |
public_key | str | "" | Public key for secure communication |
output | dict | {} | Response containing allocation details from miner |
docker_requirement | dict | Default config | Container configuration with base_image , ssh_key , ssh_port , volume_path , dockerfile |
docker_change | bool | False | Flag indicating Docker configuration changes |
docker_action | dict | Action config | Docker actions with action , ssh_key , key_type |
The deserialize()
method returns self.output
containing the allocation response from the miner.
Sources: compute/protocol.py:60-110
Challenge Protocol
Section titled “Challenge Protocol”The Challenge
class enables validators to verify miner GPU capabilities through cryptographic proof-of-work challenges, ensuring that miners have the claimed GPU resources.
Challenge Class Definition
classDiagram class bt_Synapse { +deserialize() } class Challenge { +str challenge_hash +str challenge_salt +str challenge_mode +str challenge_chars +str challenge_mask +int challenge_difficulty +dict output +deserialize() dict } bt_Synapse <|-- Challenge note for Challenge "compute/protocol.py:112-136"
Attribute | Type | Default | Description |
---|---|---|---|
challenge_hash | str | "" | Cryptographic hash to be solved |
challenge_salt | str | "" | Salt value for the challenge |
challenge_mode | str | "" | Challenge type (hashcat mode) |
challenge_chars | str | "" | Character set for brute force |
challenge_mask | str | "" | Mask pattern for challenge |
challenge_difficulty | int | compute.pow_min_difficulty | Difficulty level of the challenge |
output | dict | {} | Response containing solution or error |
The deserialize()
method returns self.output
which contains either the password solution or error information:
{"password": None, "error": f"Hashcat execution failed with code {process.returncode}: {stderr}"}
Sources: compute/protocol.py:112-136
Custom Axon and Subtensor
Section titled “Custom Axon and Subtensor”NI Compute extends Bittensor’s communication infrastructure with customized components to support the specific needs of the compute subnet.
ComputeSubnetAxon
Section titled “ComputeSubnetAxon”The ComputeSubnetAxon
class extends bittensor.core.axon.Axon
to provide compute subnet-specific functionality, including customized version tracking and middleware processing.
ComputeSubnetAxon Request Processing Flow
sequenceDiagram participant Validator participant ComputeSubnetAxon participant ComputeSubnetAxonMiddleware participant SynapseProtocol["Specs/Allocate/Challenge"] participant MinerLogic["Miner Handler Functions"] Validator->>ComputeSubnetAxon: "HTTP POST /{protocol_name}" ComputeSubnetAxon->>ComputeSubnetAxonMiddleware: "process_request()" ComputeSubnetAxonMiddleware->>ComputeSubnetAxonMiddleware: "preprocess(request)" Note over ComputeSubnetAxonMiddleware: "request.url.path.split('/')[1]"<br/>"Extract protocol name" ComputeSubnetAxonMiddleware->>ComputeSubnetAxonMiddleware: "request_synapse.from_headers()" ComputeSubnetAxonMiddleware->>ComputeSubnetAxonMiddleware: "Fill axon.nonce, axon.signature" ComputeSubnetAxonMiddleware->>SynapseProtocol: "Forward to registered handler" SynapseProtocol->>MinerLogic: "Execute miner logic" MinerLogic->>SynapseProtocol: "Fill synapse response fields" SynapseProtocol->>ComputeSubnetAxonMiddleware: "Return completed synapse" ComputeSubnetAxonMiddleware->>ComputeSubnetAxon: "Return HTTP response" ComputeSubnetAxon->>Validator: "Return synapse response"
Key Methods and Attributes:
Method/Attribute | Description |
---|---|
__init__() | Initializes with custom middleware class ComputeSubnetAxonMiddleware |
info() | Returns bittensor.AxonInfo with get_local_version() and compute-specific placeholders |
middleware_cls | Set to ComputeSubnetAxonMiddleware |
The info()
method returns customized axon information including:
version
: Fromget_local_version()
protocol
:4
placeholder1
:1
placeholder2
:2
Sources: compute/axon.py:285-390
ComputeSubnetSubtensor
Section titled “ComputeSubnetSubtensor”The ComputeSubnetSubtensor
class extends bittensor.core.subtensor.Subtensor
with additional functionality for Prometheus metrics extrinsic submission.
Key Methods:
Method | Parameters | Description |
---|---|---|
serve_prometheus() | wallet , port , netuid , wait_for_inclusion , wait_for_finalization | Serves Prometheus metrics by submitting extrinsic to blockchain |
do_serve_prometheus() | wallet , call_params , wait_for_inclusion , wait_for_finalization | Core method for submitting Prometheus extrinsics with retry logic |
Prometheus Extrinsic Flow:
sequenceDiagram participant Miner participant ComputeSubnetSubtensor participant SubstrateInterface["self.substrate"] participant Blockchain["Bittensor Blockchain"] Miner->>ComputeSubnetSubtensor: "serve_prometheus(wallet, port, netuid)" ComputeSubnetSubtensor->>ComputeSubnetSubtensor: "do_serve_prometheus()" ComputeSubnetSubtensor->>ComputeSubnetSubtensor: "@retry(delay=1, tries=3, backoff=2)" ComputeSubnetSubtensor->>SubstrateInterface: "compose_call('SubtensorModule', 'serve_prometheus')" ComputeSubnetSubtensor->>SubstrateInterface: "create_signed_extrinsic(call, wallet.hotkey)" ComputeSubnetSubtensor->>SubstrateInterface: "submit_extrinsic()" SubstrateInterface->>Blockchain: "Submit prometheus extrinsic" Blockchain->>SubstrateInterface: "Extrinsic response" SubstrateInterface->>ComputeSubnetSubtensor: "response.process_events()" ComputeSubnetSubtensor->>Miner: "return (success, error_message)"
The implementation includes retry logic with exponential backoff and comprehensive error handling for substrate communication failures.
Sources: compute/axon.py:152-283
ComputeSubnetAxonMiddleware
Section titled “ComputeSubnetAxonMiddleware”The ComputeSubnetAxonMiddleware
class extends bittensor.core.axon.AxonMiddleware
and handles the preprocessing of requests, extracting the appropriate Synapse protocol based on the URL path and preparing it for execution.
Middleware Processing Implementation:
flowchart TD Request["HTTP Request"] --> ExtractName["request.url.path.split('/')[1]"] ExtractName --> GetSynapse["self.axon.forward_class_types.get(request_name)"] GetSynapse --> CreateSynapse["request_synapse.from_headers(request.headers)"] CreateSynapse --> FillAxon["synapse.axon.__dict__.update()"] FillAxon --> FillDendrite["synapse.dendrite.__dict__.update()"] FillDendrite --> Sign["self.axon.wallet.hotkey.sign(message)"] Sign --> ReturnSynapse["return synapse"] GetSynapse -->|"None"| UnknownSynapseError["raise UnknownSynapseError"] CreateSynapse -->|"Exception"| SynapseParsingError["raise SynapseParsingError"] ExtractName -->|"Exception"| InvalidRequestNameError["raise InvalidRequestNameError"]
Key preprocess()
Method Logic:
- Request Name Extraction:
request.url.path.split("/")[1]
- Synapse Type Resolution:
self.axon.forward_class_types.get(request_name)
- Synapse Creation:
request_synapse.from_headers(request.headers)
- Axon Info Population:
version
:__version_as_int__
uuid
:str(self.axon.uuid)
nonce
:time.monotonic_ns()
status_code
:"100"
- Dendrite Info Population: Client
port
andip
- Signature Generation:
message = f"{synapse.axon.nonce}.{synapse.dendrite.hotkey}.{synapse.axon.hotkey}.{synapse.axon.uuid}"synapse.axon.signature = f"0x{self.axon.wallet.hotkey.sign(message).hex()}"
Sources: compute/axon.py:391-487
Communication Flow
Section titled “Communication Flow”The following diagram illustrates the end-to-end communication flow between validators and miners using the three core protocols.
Complete Protocol Communication Flow
sequenceDiagram participant Validator["neurons/validator.py"] participant Dendrite["bt.dendrite"] participant MinerAxon["ComputeSubnetAxon"] participant Middleware["ComputeSubnetAxonMiddleware"] participant MinerLogic["neurons/miner.py"] Note over Validator,MinerLogic: 1. Hardware Discovery (Specs Protocol) Validator->>Dendrite: "query(Specs(specs_input=''))" Dendrite->>MinerAxon: "POST /Specs" MinerAxon->>Middleware: "preprocess(request)" Middleware->>MinerLogic: "forward_fn(Specs)" MinerLogic->>MinerLogic: "Get hardware specifications" MinerLogic->>Middleware: "specs_output=hardware_json" Middleware->>MinerAxon: "return synapse" MinerAxon->>Dendrite: "HTTP 200 + specs_output" Dendrite->>Validator: "synapse.deserialize()" Note over Validator,MinerLogic: 2. GPU Verification (Challenge Protocol) Validator->>Dendrite: "query(Challenge(challenge_hash, challenge_difficulty))" Dendrite->>MinerAxon: "POST /Challenge" MinerAxon->>Middleware: "preprocess(request)" Middleware->>MinerLogic: "forward_fn(Challenge)" MinerLogic->>MinerLogic: "Execute hashcat proof-of-work" MinerLogic->>Middleware: "output={'password': solution}" Middleware->>MinerAxon: "return synapse" MinerAxon->>Dendrite: "HTTP 200 + output" Dendrite->>Validator: "synapse.deserialize()" Note over Validator,MinerLogic: 3. Resource Allocation (Allocate Protocol) Validator->>Dendrite: "query(Allocate(timeline, device_requirement, docker_requirement))" Dendrite->>MinerAxon: "POST /Allocate" MinerAxon->>Middleware: "preprocess(request)" Middleware->>MinerLogic: "forward_fn(Allocate)" MinerLogic->>MinerLogic: "Create Docker container" MinerLogic->>Middleware: "output={'ssh_address': address, 'ssh_port': port}" Middleware->>MinerAxon: "return synapse" MinerAxon->>Dendrite: "HTTP 200 + output" Dendrite->>Validator: "synapse.deserialize()"
Sources: compute/protocol.py:1-136 , compute/axon.py:391-487
Protocol Version Management
Section titled “Protocol Version Management”NI Compute implements version tracking through the __version_as_int__
global variable and get_local_version()
function to ensure protocol compatibility between nodes.
Version Integration Points
flowchart TD version_as_int["__version_as_int__"] --> axon_info["ComputeSubnetAxon.info()"] version_as_int --> serve_params["AxonServeCallParams.version"] version_as_int --> middleware["ComputeSubnetAxonMiddleware.preprocess()"] get_local_version["get_local_version()"] --> axon_info axon_info --> bittensor_AxonInfo["bittensor.AxonInfo"] serve_params --> custom_serve_extrinsic["custom_serve_extrinsic()"] middleware --> synapse_axon["synapse.axon.version"] bittensor_AxonInfo --> blockchain["Blockchain Registration"] custom_serve_extrinsic --> blockchain
Version Usage in Communication:
Component | Usage | Code Reference |
---|---|---|
ComputeSubnetAxonMiddleware.preprocess() | Sets synapse.axon.version = __version_as_int__ | compute/axon.py:470 |
ComputeSubnetAxon.info() | Returns bittensor.AxonInfo(version=get_local_version()) | compute/axon.py:379 |
custom_serve_extrinsic() | Uses version=__version_as_int__ in AxonServeCallParams | compute/axon.py:104 |
The version is encoded as an integer for efficient blockchain storage and comparison, ensuring that all communication includes version information for compatibility checking.
Sources: compute/axon.py:47-104 , compute/axon.py:376-388 , compute/axon.py:470
Subtensor Integration
Section titled “Subtensor Integration”NI Compute extends the standard Bittensor Subtensor to support custom extrinsics required for the compute subnet through the custom_serve_extrinsic
function and ComputeSubnetSubtensor
class.
Custom Serve Extrinsic Implementation
flowchart TD custom_serve_extrinsic["custom_serve_extrinsic()"] --> unlock_key["unlock_key(wallet, 'hotkey')"] unlock_key --> create_params["AxonServeCallParams()"] create_params --> get_neuron["subtensor.get_neuron_for_pubkey_and_subnet()"] get_neuron --> check_updated{"neuron_up_to_date?"} check_updated -->|"True"| return_true["return True"] check_updated -->|"False"| do_serve["do_serve_axon()"] do_serve --> wait_check{"wait_for_inclusion or wait_for_finalization?"} wait_check -->|"True"| check_success{"success?"} wait_check -->|"False"| return_true2["return True"] check_success -->|"True"| return_true3["return True"] check_success -->|"False"| return_false["return False"]
Key Customizations:
- Custom Version Integration: Uses
__version_as_int__
from the compute subnet - Modified Parameters: Includes compute-specific
placeholder1
andplaceholder2
values - Enhanced Logging: Custom debug messages for axon serving status
- Registration Override: Replaces
bittensor.core.extrinsics.serving.serve_extrinsic
with custom implementation
AxonServeCallParams Structure:
params = AxonServeCallParams( version=__version_as_int__, # Compute subnet version ip=net.ip_to_int(ip), # IP address as integer port=port, # Port number ip_type=net.ip_version(ip), # IP version (4 or 6) netuid=netuid, # Subnet ID hotkey=wallet.hotkey.ss58_address, # Hotkey address coldkey=wallet.coldkeypub.ss58_address, # Coldkey address protocol=protocol, # Protocol version placeholder1=placeholder1, # Reserved field placeholder2=placeholder2, # Reserved field certificate=certificate, # TLS certificate)
The system patches the global Bittensor extrinsic function:
bittensor.core.extrinsics.serving.serve_extrinsic = custom_serve_extrinsic
Sources: compute/axon.py:63-150
Summary
Section titled “Summary”The communication protocols in NI Compute form a critical infrastructure layer that enables secure, efficient interaction between validators and miners in the decentralized GPU marketplace. By extending Bittensor’s core communication components, NI Compute implements specialized protocols for hardware discovery, verification, and resource allocation.
This layered approach ensures:
- Discoverability: Validators can discover and verify miner hardware capabilities
- Security: Secure communication channels for sensitive operations
- Resource Management: Efficient allocation and management of GPU resources
- Monitoring: Integration with Prometheus for metrics collection
Understanding these protocols is essential for developing and extending the NI Compute platform with new features and capabilities.