WIFI LLM Application

Note

• SDK path: http://gitlab.nationalchip.com/nationalchip/voice-wifi-solution/ovp_aiot
• App path: app/smartbot_app/ under the SDK path
• Applicable chips: 8005/8006

Description

• This application is closely related to the WIFI LLM solution. It is recommended to first read the WIFI LLM Solution Overview.

1. Overview

The WIFI LLM application works with the WIFI module via the UART protocol. The MCU handles voice algorithms (KWS, VAD), while the WIFI module handles network connectivity and LLM business logic. All events (wake-up, VAD, audio data) are reported to the WIFI module via the UART protocol, and the WIFI module sends commands to control playback, wake-up status, etc.

2. Features

• UART Protocol Communication: Bidirectional command/response protocol for exchanging events and commands with the WIFI module
• Keyword Wake-up Bridging: KWS events forwarded to the WIFI module, which decides activation/sleep status
• VAD Bridging: VAD status reported to the WIFI module in real-time
• Delayed Wake-up: Configurable delay before activation after wake word detection (default 1000ms)
• Timeout Management: Configurable active window (default 10s), automatically returns to listening state after timeout
• UART Audio Transmission: Supports Opus/PCM format speaker audio streaming with ACK/GPIO flow control
• I2S Audio Transmission: Optional I2S audio input/output
• Microphone Upload: Uploads microphone data to the WIFI module
• Firmware Upgrade: OTA firmware upgrade via UART
• Factory Testing: Audio loopback, recording playback, GPIO tests
• Self-activation Filtering: Prevents false wake-up during MCU self-playback
• LED Indication: VAD LED + status LED

3. Directory Structure

app/smartbot_app/
├── smartbot_app.c            # Main program (event dispatch, state machine)
├── app.mk                     # Build configuration
├── app.name                   # Kconfig option
├── Kconfig                    # Sub-configuration (timeout, UART, LED, etc.)
├── led/                       # LED indicators
│   ├── led.c / led.h
├── timeout_manager/           # Timeout manager
│   ├── timeout_manager.c / .h
├── vad/                       # Voice activity detection
│   ├── vad.c / vad.h
└── uart_protocol/             # UART protocol stack
    ├── uart_protocol.c / .h
    ├── user_kws/              #   User-defined keywords
    ├── mic_upload/            #   Microphone upload
    ├── spk/                   #   Speaker audio (UART / I2S)
    │   ├── uart_spk/          #     UART speaker
    │   │   ├── flow_ctrl/     #       Flow control (ACK / GPIO)
    │   │   ├── spk_buf_manager/#     Buffer management
    │   │   └── uart_spk_player/#     Player (Opus/PCM parsing)
    │   └── i2s_spk/           #     I2S speaker
    ├── factory_test/          #   Factory testing
    ├── upgrade/               #   OTA upgrade
    ├── data_query/            #   Status query
    ├── voice_config/          #   Voice configuration
    ├── filter_self_act/       #   Self-activation filtering
    └── utility/               #   General utilities

4. Configuration

4.1 Top-level Configuration

Configuration Item	Type	Default	Description
CONFIG_APP_KWS_TIME_OUT	int	10	KWS wake-up timeout (seconds)
CONFIG_DELAY_AWAKE_DELAY_MS	int	1000	Delay before activation after wake-up (ms), ≤0 activates immediately
CONFIG_ENABLE_REPORT_MCU_PLAY_STATUS	bool	y	Report playback status to the WIFI module
CONFIG_ENABLE_RESET_VAD_ON_MIC_OPEN	bool	y	Reset VAD when MIC is opened
CONFIG_ENABLE_MIC_AUTO_TOGGLE	bool	n	Automatically enable MIC on wake-up, disable on timeout
CONFIG_ENABLE_WAKE_SKIP_PLAY_AUDIO_CHECK	bool	n	Skip self-playback audio detection (allows KWS to work during self-playback)
CONFIG_DISABLE_WAKE_TIMEOUT	bool	n	Debug: disable wake-up timeout

4.2 UART Communication Configuration

Configuration Item	Type	Default	Description
CONFIG_SMARTBOT_UART	int	0	UART interface number (0-2)
CONFIG_SMARTBOT_UART_BAUDRATE	int	1000000	UART baud rate
CONFIG_ENABLE_UART_TRANSMISSION	bool	y	Enable UART audio transmission

4.3 Speaker Configuration

Configuration Item	Type	Default	Description
CONFIG_ENABLE_FORMAT_OPUS	bool	—	Speaker audio format: Opus (default)
CONFIG_ENABLE_FORMAT_PCM	bool	—	Speaker audio format: PCM
CONFIG_ENABLE_ACK_FLOW_CTRL	bool	—	ACK flow control (default)
CONFIG_ENABLE_GPIO_FLOW_CTRL	bool	—	GPIO flow control
CONFIG_SPK_FLOW_CTRL_PIN	int	9	GPIO flow control pin
CONFIG_PLAY_MUTE_DELAY_TIME	int	30	Delay before muting after playback ends (seconds)

4.4 Microphone Upload Configuration

Configuration Item	Type	Default	Range	Description
CONFIG_VAD_FALLBACK_FRAMES	int	22	15-35	VAD fallback frame count
CONFIG_MIC_FALLBACK_DATA_CONTINUOUS_UPLOAD	bool	n	Continuously upload microphone fallback data

4.5 I2S Transmission Configuration

Configuration Item	Type	Default	Description
CONFIG_ENABLE_I2S_TRANSFER	bool	n	Enable I2S audio transmission (mutually exclusive with UART transmission)
CONFIG_CACHE_TIME_MS	int	840	I2S cache time (ms)
CONFIG_ENABLE_I2S_RX_M_TX_M	bool	—	RX master + TX master mode (default)
CONFIG_ENABLE_I2S_RX_S_TX_S	bool	—	RX slave + TX slave mode

4.6 LED Configuration

Configuration Item	Type	Default	Description
CONFIG_ENABLE_LED	bool	y	Enable LED
CONFIG_APP_LED_VAD_PIN	int	6	VAD LED GPIO
CONFIG_APP_LED_STATUS_PIN	int	13	Status LED GPIO
CONFIG_APP_LED_ON_LEVEL	int	1	LED on level (0: active low, 1: active high)

5. Working Principle

5.1 Overall Architecture

  ┌──────────────┐        UART Protocol       ┌──────────────┐
  │   MCU (GX8006)│ ◄──────────────────► │  WIFI Module   │
  │              │                        │              │
  │  KWS / VAD   │ ──Event Report──►     │  Business Logic│
  │  Audio Processing│ ◄──Command Dispatch── │  Playback Control│
  │  Mic Upload  │ ──Audio Data──►       │  Status Management│
  └──────────────┘                        └──────────────┘

5.2 Initialization Flow

1. Read wake-up source
2. Enable VPA mode auto-switching
3. Initialize timeout manager (default 10s), initial state is TIMEOUT_ACTIVE (wake-up allowed)
4. Initialize UART protocol stack, establish communication with the WIFI module
5. Initialize LED, turn on status LED

5.3 State Machine

        Wake Word Detection / WIFI Module Enters Wake-up
LISTEN_STATE ────────────────────→ ACTIVE_STATE
     ↑                                  │
     │                                  │ Timeout / WIFI Module Exits Wake-up
     └──────────────────────────────────┘

• LISTEN_STATE: Waiting for wake word or WIFI module command
• ACTIVE_STATE: Awakened, VAD active, audio processing running

Timeout manager semantics (note the naming):

Timeout State	Actual Meaning
TIMEOUT_ACTIVE	Idle/timeout expired, wake-up allowed again
TIMEOUT_INACTIVE	System is within the active window, not yet timed out

5.4 Delayed Wake-up

After wake word detection, activation is not immediate; a one-shot timer is registered:

1. Receive KWS wake-up event
2. Wait for CONFIG_DELAY_AWAKE_DELAY_MS (default 1000ms)
3. Timer triggers → set ACTIVE_STATE → initialize VAD

5.5 Event Handling

Event	Handling Logic
EVENT_AUDIO_IN_RECORD_DONE	Forward to UART protocol, WIFI module can obtain raw audio
EVENT_VAD_STATUS	Update SPL, adjust VAD sensitivity based on playback status, forward to WIFI module
EVENT_WAKEUP_ENTER_RX	WIFI module command to enter wake-up: reset timeout, set ACTIVE_STATE
EVENT_WAKEUP_EXIT_RX	WIFI module command to exit wake-up: set LISTEN_STATE
KWS wake-up event	Set ACTIVE_STATE after delay, forward to WIFI module

5.6 Main Loop

app_task_loop() calls uart_protocal_loop() to handle UART transmission and reception.

6. Usage

6.1 Compilation

Configuration files are located in configs/lightning/, select based on transmission method:

Configuration File	Description
lightning_vad_028_kws_021_ack_flowctrl.config	UART transmission + ACK flow control
lightning_vad_028_kws_021_io_flowctrl.config	UART transmission + GPIO flow control
lightning_vad_028_kws_021_i2s.config	I2S transmission

cp configs/lightning/lightning_vad_028_kws_021_ack_flowctrl.config .config
make defconfig
make clean; make

6.2 Flashing and Running

cd tools/bootx/
./flash_nor.sh 0 -r 1000000

6.3 WIFI Module Coordination

This App requires the WIFI module to work in coordination via the UART protocol. After the MCU-side configuration is complete, the WIFI module side must implement the corresponding UART protocol stack.

7. Expected Output

1. After power-on, the status LED lights up, waiting for WIFI module communication to be ready
2. Speak the wake word → MCU reports KWS event → WIFI module decides whether to activate
3. WIFI module sends playback command → MCU plays audio via UART/I2S
4. Timeout with no activity → MCU automatically returns to listening state, reports timeout event

8. Notes

1. Dependency on WIFI Module: This App does not include business logic; it must be used with a WIFI module, and the UART protocol stack must be implemented on the WIFI module side
2. Abort on UART Initialization Failure: If UART protocol stack initialization fails, app_init() returns an error directly
3. UART and I2S are Mutually Exclusive: CONFIG_ENABLE_UART_TRANSMISSION and CONFIG_ENABLE_I2S_TRANSFER cannot be enabled simultaneously
4. Timeout Manager Naming: TIMEOUT_INACTIVE means within the active window (not timed out), TIMEOUT_ACTIVE means the timeout has expired — the naming is counterintuitive
5. Playback Prevents Timeout: When WIFI module-driven playback is in progress, the timeout timer is automatically reset to prevent false timeout during playback
6. VAD Can Independently Wake Up: When the wake-up switch is off but the VAD switch is on, VAD detection can directly trigger system wake-up
7. Delayed Wake-up: The default 1-second delay can be adjusted in Kconfig; setting it to 0 activates immediately
8. Sleep/Wake-up Not Implemented: The current SDK does not support sleep functionality; app_suspend() and app_resume() can remain as empty implementations