Benchmark

Edge Compute Benchmark

ClearBlade’s goal is to empower customers with all the tools needed to make an informed decision about their solution. Edge Computer Benchmark is one such method to help customers choose the right gateways for the right solution. This is accomplished by:

  • Establishing ClearBlade IoT Key Performance Metrics(KPI)
  • Tests that measure these KPI

Customer Questions Answered

  • Does this gateway run ClearBlade Software?
  • How does this gateway compare to other gateways running software via final score?
  • How much can this gateway with ClearBlade do for me? Example: publishing number of messages per second

Currently benchmarked gateways

Test Details

Types of Performance Operations

There are three types of performance operations relevant to hardware testing are performed:

  • Unit-Level Operation
    • Single operation is tested
    • Example: MQTT publish, database write
  • System-Level Operation
    • system-level operations
    • Example: Read device row, then update, then write back to db
  • Simulation-Level Operation
    • use-case oriented
    • Example: Connected Job Site, Smart Building

Unit-Level Operations

Name Desc Operation
PublishSubscribeVolume Publish high volume, small payload an MQTT message to a channel with one worker and wait on that message with another worker. Publish
PublishSubscribePayload Publish low volume, large payload …
PublishTriggerSubscribe Publish an MQTT message to a channel, launching a trigger that launches a code service that publishes a message. Another worker waits on this message. Execute
CreateCollectionRow REST, call a code service to create a row in a collection Create
UpdateCollectionRow REST, call a code service to update a row in a collection Update
ReadCollectionRow REST, call a code service to fetch a row from a collection Read
DeleteCollectionRow REST, call a code service to delete a row from a collection Delete
CryptoHash REST, call a code service to hash a long string into sha1, md5, sha256, and sha512 digest
RestructureData REST, call a code service to convert a string to an array to a map and back again transform
MachineLearning REST, neural net from signature science Process
Geo REST, call a code service to compute the distance between two points, compute whether a point is within a polygon, and compute the distance to the nearest point out of a list of points Process
CreateDevice REST, creates a device in the Device Table Create
ReadDevice REST, reads a device from the Device Table Read
UpdateDevice REST, updates a device in the Device Table Update
DeleteDevice REST, deletes a device from the Device Table Delete
StatisticalComputation jstat-statistics-toolkit for intense data computation Process

System-Level Operations

Name Maps to Unit Description
Large Scale Messaging PublishSubscribeVolumePublishSubscribePayload Publish small and large messages to many different channels and wait on many different channels.
Database Operations CreateCollectionRowReadCollectionRowUpdateCollectionRowDeleteCollectionRow Create a collection row, delete a collection row, read a collection row, and update a collection row.
Artificial Intelligence MachineLearningStatisticalComputationRestructureData Run many different instances of machine learning.
Geospatial Computation GeoRestructureDataStatisticalComputationPublishSubscribeVolume Compute many different geospatial data.
Cryptography CryptoHashRestructureData Create many different hashes and digest them repeatedly.
Device Operations CreateDeviceReadDeviceUpdateDeviceDeleteDevicePublishSubscribeVolume Create a device, delete a device, read a device, and update a device.
Anomaly Detection StatisticalComputationMachineLearning

Simulation-Level Operations

Name Maps to System Description (Ability to perform…)
Connected Job Site Large Scale MessagingGeospatial ComputationAnomaly DetectionDevice Operations Applications to connect your construction job site tools, equipment and people to your operations and business systemsLarge scale connectivity for Connected Job Site assets, such as Personnel, Tools, Machinery, Vehicles High-frequency device state managementAnomaly detection with alerts
Smart Building Large Scale MessagingGeospatial ComputationAnomaly DetectionDevice OperationsArtificial Intelligence Aggregation of disparate building systems and infrastructure into a unified view with built-in security. Real-time Edge processing utilizes Machine Learning to respond automatically to air quality, energy, temperature, traffic patterns, people movement, and parking to create an enhanced experience for tenants and facility managers. Large scale connectivity for Smart Building systems, such as HVAC, Safety, Security, Personnel, LightingGeofencing and Geospatial computation for asset tracking through facilityAnomaly detection for activity and sensor feedsArtificial Intelligence for operational optimizations and preventative maintenace
Blockchain Large Scale MessagingCryptography Application of modern cryptography tools to engage with blockchain networksHighly-concurrent, CPU-intensive cryptography workloadsLarge scale messagingProof-of-work based verification tasks
Smart City Large Scale MessagingGeospatial ComputationAnomaly DetectionDevice OperationsArtificial Intelligence Integration of city-wide operational systemsLarge scale messaging for Traffic Monitoring, Energy ConsumptionGeofencing and Geospatial computation for GPS of assetsAnomaly detection across sensor networksArtificial Intelligence for operational optimizations
Smart Factory Large Scale MessagingGeospatial ComputationAnomaly DetectionDevice OperationsArtificial Intelligence Monitor and react to working conditions, industrial processes, environmental emissions and storage conditions, and integrate with MES, ERP and CRM business systemsLarge scale messaging and connectivityGeofencing and Geospatial computation for assets through factoryPreventative Maintenance through AI and Anomaly DetectionHigh-frequency device state management
Real-Time Logistics Large Scale MessagingGeospatial ComputationDevice Operations Real-time logistics provides administration and monitoring across multi-modal routes across the world. Large scale messagingGeofencing and Geospatial computation across GPS systemsHigh-frequency device state management
Predictive Maintenance Large Scale MessagingAnomaly DetectionArtificial Intelligence Processing multiple, high-throughput data fed from machines and situational activity across sensor networks and equipment retrofitted with IoT hardware. Large scale messagingPreventative Maintenance through AI and Anomaly Detection