Subnet 27: Compute

Permission-less Compute Market
Subnet 27: Bittensor’s Layer-0, the permission-less compute market for platform composable cloud services. An abundant era in decentralized, efficient and scalable computing has begun.
With the proliferation of cloud platforms, there's a need for a subnet that can seamlessly integrate these platforms, allowing for efficient resource sharing and allocation. This compute-composable subnet will enable miners to contribute computational power, with validators ensuring the integrity and efficiency of the shared resources.
Compute-composable subnet. Integrated into various cloud platforms (e.g., Runpod, Lambda, AWS) Subnet 27 becomes a cohesive unit, enabling higher-level cloud platforms to offer seamless compute composability across different underlying platforms.
Compute is a fundamental necessity for all operations, welcome to Bittensor's Layer-0.

Decentralized Compute

Compute markets of today consist of siloed pools of compute resources separated by each provider and its designated apps to access each pool of compute. Subnet 27 decentralizes computing resources by combining siloed pools of compute on a blockchain to be validated and accessed trustlessly. This opens a door to scalable compute without the constraints of centralized power.
Subnetwork 27
Decentralized cloud computing is an innovative approach that leverages a peer-to-peer network to distribute computational resources across multiple nodes, rather than relying on centralized data centers. With the proliferation of cloud platforms, there's a need for a Bittensor subnet that can seamlessly integrate these platforms, allowing for efficient resource sharing and allocation.

Subnet 27 Overview

Miners

Miners contribute processing resources, notably GPU (Graphics Processing Unit) and CPU (Central Processing Unit) instances, to facilitate optimal performance in essential GPU and CPU-based computing tasks.
Dynamic Reservation Timelines: A key aspect of the miners' contribution is the management of resource reservations. Miners have the autonomy to set specific timelines for each reservation of their computational resources. This timeline dictates the duration for which the resources are allocated to a particular task or user. Once the set timeline reaches its conclusion, the reservation automatically expires, thereby freeing up the resources for subsequent allocations. This mechanism ensures a dynamic and efficient distribution of computational power, catering to varying demands within the network.
Performance-Based Mining: The system operates on a performance-based reward mechanism, where miners are incentivized through a tiered reward structure correlated to the processing capability of their hardware. High-performance devices are eligible for increased compensation, reflecting their greater contribution to the network's computational throughput. Emphasizing the integration of GPU instances is critical due to their superior computational power, particularly in tasks demanding parallel processing capabilities.
Consequently, miners utilizing GPU instances are positioned to receive substantially higher rewards compared to their CPU counterparts, in alignment with the greater processing power and efficiency GPUs bring to the network.

Validators

Validators hold the critical responsibility of rigorously assessing and verifying the computational capabilities of miners. This multifaceted evaluation process commences with validators requesting miners to provide comprehensive performance data, which includes not only processing speeds and efficiencies but also critical metrics like Random Access Memory (RAM) capacity and disk space availability.
Compute Data: The inclusion of RAM and disk space measurements is vital, as these components significantly impact the overall performance and reliability of the miners' hardware. RAM capacity influences the ability to handle large or multiple tasks simultaneously, while adequate disk space ensures sufficient storage.
Computational Integrity: Following the receipt of this detailed hardware and performance information, validators proceed to test the miners' computational integrity. This is achieved by presenting them with complex hashing challenges, designed to evaluate the processing power and reliability of the miners' systems. Validators adjust the difficulty of these problems based on the comprehensive performance profile of each miner, including their RAM and disk space metrics.
In addition to measuring the time taken by miners to resolve these problems, validators meticulously verify the accuracy of the responses. This thorough examination of both speed and precision, complemented by the assessment of RAM and disk space utilization, forms the crux of the evaluation process.
Scoring Mechanism: Validators update the miners' scores, reflecting a holistic view of their computational capacity, efficiency, and hardware quality. This score then determines the miner's weight within the network, directly influencing their potential rewards and standing.
This scoring process, implemented through a Python script, considers various factors including CPU, GPU, hard disk, and RAM performance. The script's structure and logic are outlined below:
  1. 1.
    Score Calculation Function:
    • The score function aggregates performance data from different hardware components.
    • It calculates individual scores for CPU, GPU, hard disk, and RAM.
    • These scores are then weighted and summed to produce a total score.
    • A registration bonus is applied if the miner is registered, enhancing the total score.
  2. 2.
    Component-Specific Scoring Functions:
    • get_cpu_score, get_gpu_score, get_hard_disk_score, and get_ram_score are functions dedicated to calculating scores for each respective hardware component.
    • These functions consider the count, frequency, capacity, and speed of each component, applying a specific level value for normalization.
    • The scores are derived based on the efficiency and capacity of the hardware.
  3. 3.
    Weight Assignment:
    • The script defines weights for each hardware component's score, signifying their importance in the overall performance.
    • GPU has the highest weight (0.55), reflecting its significance in mining operations.
    • CPU, hard disk, and RAM have lower weights (0.2, 0.1, and 0.15, respectively).
  4. 4.
    Score Aggregation:
    • The individual scores are combined into a numpy array, score_list.
    • The weights are also arranged in an array, weight_list.
    • The final score is calculated using a dot product of these arrays, multiplied by 10, and then adjusted with the registration bonus.
  5. 5.
    Handling Multiple CPUs/GPUs:
    • The scoring functions for CPUs (get_cpu_score) and GPUs (get_gpu_score) are designed to process data that can represent multiple units.
    • For CPUs, the count variable in cpu_info represents the number of CPU cores or units available. The score is calculated based on the cumulative capability, taking into account the total count and frequency of all CPU cores.
    • For GPUs, similar logic applies. The script can handle data representing multiple GPUs, calculating a cumulative score based on their collective capacity and speed.
  6. 6.
    CPU Scoring (Multiple CPUs):
    • The CPU score is computed by multiplying the total number of CPU cores (count) by their average frequency (frequency), normalized against a predefined level.
    • This approach ensures that configurations with multiple CPUs are appropriately rewarded for their increased processing power.
  7. 7.
    GPU Scoring (Multiple GPUs):
    • The GPU score is calculated by considering the total capacity (capacity) and the average speed (average of graphics_speed and memory_speed) of all GPUs in the system.
    • The score reflects the aggregate performance capabilities of all GPUs, normalized against a set level.
    • This method effectively captures the enhanced computational power provided by multiple GPU setups.
  8. 8.
    Aggregated Performance Assessment:
    • The final score calculation in the score function integrates the individual scores from CPU, GPU, hard disk, and RAM.
    • This integration allows the scoring system to holistically assess the collective performance of all hardware components, including scenarios with multiple CPUs and GPUs.
  9. 9.
    Implications for Miners:
  • Miners with multiple GPUs and/or CPUs stand to gain a higher score due to the cumulative calculation of their hardware's capabilities.
  • This approach incentivizes miners to enhance their hardware setup with additional CPUs and GPUs, thereby contributing more processing power to the network.
The weight assignments are as follows:
  • GPU Weight: 0.55
  • CPU Weight: 0.2
  • Hard Disk Weight: 0.1
  • RAM Weight: 0.15

Example 1: Miner A's Hardware Scores and Weighted Total

  1. 1.
    CPU Score: Calculated as (2 cores * 3.0 GHz) / 1024 / 50.
  2. 2.
    GPU Score: Calculated as (8 GB * (1 GHz + 1 GHz) / 2) / 200000.
  3. 3.
    Hard Disk Score: Calculated as (500 GB * (100 MB/s + 100 MB/s) / 2) / 1000000.
  4. 4.
    RAM Score: Calculated as (16 GB * 2 GB/s) / 200000.
Now, applying the weights:
  • Total Score = (CPU Score × 0.2) + (GPU Score × 0.55) + (Hard Disk Score × 0.1) + (RAM Score × 0.15)
  • If registered, add a registration bonus.

Example 2: Miner B's Hardware Scores and Weighted Total

  1. 1.
    CPU Score: Calculated as (4 cores * 2.5 GHz) / 1024 / 50.
  2. 2.
    GPU Score: Calculated as ((6 GB + 6 GB) * (1.5 GHz + 1.2 GHz) / 2) / 200000.
  3. 3.
    Hard Disk Score: Calculated as (1 TB * (200 MB/s + 150 MB/s) / 2) / 1000000.
  4. 4.
    RAM Score: Calculated as (32 GB * 3 GB/s) / 200000.
Applying the weights:
  • Total Score = (CPU Score × 0.2) + (GPU Score × 0.55) + (Hard Disk Score × 0.1) + (RAM Score × 0.15)
  • Since Miner B is not registered, no registration bonus is added.

Impact of Weights on Total Score

  • The GPU has the highest weight (0.55), signifying its paramount importance in mining tasks, which are often GPU-intensive. Miners with powerful GPUs will thus receive a significantly higher portion of their total score from the GPU component.
  • The CPU, while important, has a lower weight (0.2), reflecting its lesser but still vital role in the mining process.
  • The hard disk and RAM have the lowest weights (0.1 and 0.15, respectively), indicating their supportive but less critical roles compared to GPU and CPU.
Validator Requirements: It is important to note that the role of validators, in contrast to miners, does not require the integration of GPU instances. Their function revolves around data integrity and accuracy verification, involving relatively modest network traffic and lower computational demands. As a result, their hardware requirements are less intensive, focusing more on stability and reliability rather than high-performance computation.

Resource Allocation Mechanism

The allocation mechanism within subnet 27 is designed to optimize the utilization of computational resources effectively. Key aspects of this mechanism include:
  1. 1.
    Resource Requirement Analysis: The mechanism begins by analyzing the specific resource requirements of each task, including CPU, GPU, memory, and storage needs.
  2. 2.
    Miner Selection: Based on the analysis, the mechanism selects suitable miners that meet the resource requirements. This selection process considers the current availability, performance history, and network weights of the miners.
  3. 3.
    Dynamic Allocation: The allocation of tasks to miners is dynamic, allowing for real-time adjustments based on changing network conditions and miner performance.
  4. 4.
    Efficiency Optimization: The mechanism aims to maximize network efficiency by matching the most suitable miners to each task, ensuring optimal use of the network's computational power.
  5. 5.
    Load Balancing: It also incorporates load-balancing strategies to prevent overburdening individual miners, thereby maintaining a healthy and sustainable network ecosystem.
Through these functionalities, the allocation mechanism ensures that computational resources are utilized efficiently and effectively, contributing to the overall robustness and performance of the network.
Validators can send requests to reserve access to resources from miners by specifying the specs manually in the in register.py and running this script: https://github.com/neuralinternet/Compute-Subnet/blob/main/neurons/register.py for example: {'cpu':{'count':1}, 'gpu':{'count':1}, 'hard_disk':{'capacity':10737418240}, 'ram':{'capacity':1073741824}}
Official Subnet 27 Channel: https://discord.gg/bittensor

Powered By Bittensor

Subnet 27 brings an entirely new resource to Bittensor, compute. Arguably, the most important and finite resource needed for the creation of machine intelligence. All network participants will have access to an ever-expanding pool of compute for all development needs.
Governments and regulatory bodies are in the process of regulating GPUs for AI. These political moves coupled with a shortage of GPUs in the market hinder the collective when it comes to AI/ML development and access. Big tech and those with deep pockets are the only ones that can participate in this transformative technology.
Subnet 27 changes this. With a decentralized compute subnet plugged into Bittensor, we become ungovernable. End of the day, what is a decentralized supercomputer without access to permissionless compute?
Subnetwork 27