Skip to Content

Data Modalities

Both Hailo-8 and DeepX AI accelerators are designed for high-performance, low-power inference at the edge. This means they are optimized to process data after an AI model has been trained. While the training process often involves high-precision floating-point numbers (FP32), these edge accelerators typically leverage lower precision data types for efficient inference.

Here's a breakdown of the appropriate data modalities for Hailo-8 and DeepX, which generally align given their shared purpose in edge AI:

Primary Data Modalities (Well-Suited for Hailo-8 and DeepX)

These accelerators are primarily optimized for deep learning workloads, making them excel at:

  1. Image Data (Computer Vision): This is arguably the most common and impactful modality for these accelerators.
    • Applications: Object detection (e.g., YOLO, SSD), image classification, semantic segmentation, instance segmentation, pose estimation, facial recognition, depth estimation, quality inspection, anomaly detection in visual data, security and surveillance.
    • Data Characteristics: Raw pixel data from cameras or stored image files. Often preprocessed into specific resolutions (e.g., 640x480, 1280x720) and color formats (RGB, BGR, YUV) before being fed to the model.
  2. Numerical/Sensor Data (Time-Series & Structured Data): While less visually intuitive than images, processing numerical data streams is critical for many industrial and IoT applications.
    • Applications: Predictive maintenance (vibration, temperature, current, pressure data), anomaly detection in sensor readings, environmental monitoring, motion analysis (accelerometer, gyroscope data), industrial control, energy management.
    • Data Characteristics: Tabular data where each row is a sample and columns represent different sensor readings, measurements, or calculated features. This often involves time-series data, where the sequence of values over time is important.
  3. Audio Data (Speech & Sound Processing):
    • Applications: Keyword spotting, voice commands, speaker recognition, audio event detection (e.g., glass breaking, alarms), environmental sound classification.
    • Data Characteristics: Raw audio waveforms, often converted into spectrograms or other frequency-domain representations for deep learning models.

Emerging and Supported Modalities (with increasingly robust support)

As AI technology evolves and these accelerators become more versatile, their support extends to:

  1. Multi-Modal Data: This involves combining data from two or more modalities to gain richer insights.
    • Applications: Autonomous driving (combining camera vision with LiDAR/radar data), robotics (vision + tactile sensors), smart home devices (audio + motion detection), human-computer interaction (vision + gesture + voice).
    • Data Characteristics: A synchronized input stream from multiple sensor types, often requiring fusion techniques (early fusion, late fusion) within the AI model architecture.
  2. Text Data (Natural Language Processing - NLP):
    • Applications: On-device natural language understanding (NLU), keyword extraction, sentiment analysis, simple chatbots, command processing.
    • Data Characteristics: Text strings, typically tokenized and converted into numerical embeddings before being fed to the model. While LLMs traditionally require significant compute, Hailo's newer chips (like Hailo-10H) are explicitly supporting LLMs and generative AI at the edge. DeepX also mentions LLMs and multimodal AI in their capabilities.
  3. Generative AI (including Large Language Models - LLMs and Vision Language Models - VLMs):
    • Applications: On-device code generation, text summarization, content generation, conversational AI, image generation from text prompts.
    • Data Characteristics: Dependent on the specific generative model. For LLMs, it's primarily text. For VLMs, it's a combination of text and image/video data. Both Hailo and DeepX are actively developing support for these complex models on their newer generations and existing hardware.

Key Considerations for Data with Edge AI Accelerators:

  • Quantization (INT8, INT4): These accelerators achieve their high efficiency by performing inference using lower precision integer data types (e.g., 8-bit integers, 4-bit integers) rather than the 32-bit floating-point numbers often used during training. Your training process and the model compilation tools (provided by Hailo and DeepX) will handle this quantization.
  • Input Resolution and Format: Models have specific input requirements (e.g., image dimensions, number of channels). Data must be preprocessed to match these requirements.
  • Data Preprocessing: Before feeding data to the AI model on the accelerator, tasks like normalization, resizing, cropping, and color space conversion are often necessary.
  • Model Compatibility: While these accelerators support popular frameworks like TensorFlow, PyTorch, and ONNX, the specific operations and layers within your chosen model must be compatible with the accelerator's architecture. Both Hailo and DeepX provide comprehensive SDKs and model zoos to guide this.

In summary, Hailo-8 and DeepX are highly capable for a broad spectrum of AI applications at the edge. While computer vision remains a cornerstone, their capabilities extend to various numerical, audio, multi-modal, and increasingly, generative AI and NLP tasks, all optimized for efficient, low-latency inference on the Digital View AI platforms.

DeepX DX-M1