Private Technical Manual · RSAP v0.8.1 Baseline

Inside RSAP.
This is how he works.

Owner/service reference for the original RoboSapien X AI retrofit: realtime Grok voice, persistent memory, weather, curiosity, transcript-gated shutdown, Pico infrared control, hardware architecture, release discipline, and backpack-safe motion staging.

23Manual Sections
7Local AI Tools
60Pico IR Commands
Reference key: Private Owner/service detail ⚠ Safety Shutdown or movement risk ◎ IR Pico / RoboSapien control Dry Run AI movement transmission locked

1. Overview

RSAP is a WowWee Robosapien X retrofitted with a standalone Raspberry Pi/Pico AI system. The original robot body retains its native motor electronics and receives control only through infrared commands. The retrofit does not wire the Raspberry Pi directly to the motors.

Identity

Unit: RSAP-0001
Build: Original Prototype
Owner: Spiro Mandylor
Stage: Phase 1


Current capabilities

Realtime speech-to-speech conversation, persistent memory, weather and five-day forecasts, conversational probing, spontaneous curiosity, Pico diagnostics, dry-run motion planning, and voice-triggered Linux shutdown.

Not yet production-enabled

Automatic startup/service recovery, physical AI-driven gesture execution, continuous locomotion, IMU-based motion monitoring, and camera/vision.

Design rule: the backpack is a standalone removable AI/IR/audio module. Factory RoboSapien motors and motion electronics remain under factory control; the retrofit communicates with the robot by IR.

2. Safety & release state

0.8.1 safety locks

  • motion_execution_mode = dry_run
  • Automatic gestures disabled.
  • Continuous locomotion disabled.
  • IMU locomotion monitoring disabled.
  • Production motion planning does not transmit IR.
  • All 39 backpack-dependent motion commands remain physically unvalidated with the final backpack installed.

Hardware handling

  • Support RSAP against falls during future movement tests.
  • Keep hands, clothing, cables and objects clear of arm/waist travel.
  • Do not test continuous walking near stairs or table edges.
  • Do not expose the factory LISTEN routine to AI control; the original mic has been removed.
Important: voice shutdown in 0.8.1 is real. A recognized explicit shutdown phrase can halt Linux. Test shutdown only when you are ready for the Raspberry Pi to power down.

3. Normal operation

Current manual launch

0.8.1 is not yet the autostart/reliability release. Start the production runtime manually:

ssh rsap@rsap.local
cd /home/rsap/rsap
.venv/bin/python main.py --config /home/rsap/rsap/rsap_config.json --run

On a successful launch, the terminal should show the runtime start, memory counts, Grok session configuration, PipeWire streams, and a production-ready message. RSAP then speaks one randomized startup welcome. The welcome is spoken only once per service launch and should not repeat merely because the realtime connection reconnects.

Startup welcome pool

  • “RSAP online. Systems nominal. Your move, Spiro.”
  • “Back online. I assume you have a plan.”
  • “RSAP online. Try to make this interesting.”
  • “Systems are up. What are we working on?”
  • “Online again. I was enjoying the silence, but go ahead.”

Stopping the foreground runtime without powering off

From the SSH terminal use Ctrl+C. This stops the foreground Python process. It is different from RSAP's explicit voice shutdown, which halts Linux.

4. Voice commands & natural-language functions

Conversation & utility examples

  • “What time is it?”
  • “What’s the weather?”
  • “Give me the five-day forecast.”
  • “What’s the weather in Montreal?”
  • “Remember that …”
  • “What do you remember about …?”
  • Natural conversational questions and follow-ups.

Control examples

  • “RSAP, power down.”
  • “Please shut down now.”
  • “Turn yourself off.”
  • “Goodbye” / “bye” are not control commands and do not sleep or shut down RSAP.

Local tool allowlist

ToolPurpose
get_local_timeCurrent local date/time
get_system_statusRSAP runtime/system status
get_pico_statusPico/IR controller status
search_memoryRead-only memory lookup
get_weatherCurrent + five-day weather
remember_factExplicit append-only fact memory
request_system_shutdownExplicit transcript-gated Linux shutdown
Motion names may be understood and planned, but 0.8.1 still executes production motion in dry-run only. Spoken motion planning must not physically move RSAP until backpack validation is completed and execution is deliberately unlocked.

5. AI personality, probing & spontaneous curiosity

RSAP uses xAI Grok Realtime with the Cosmo voice. Personality targets are blunt, calm, competent, candid and dryly humorous rather than sentimental. Current configuration sets humor to 70% and candor to 95%.

Conversational probing

Existing 0.8 behavior remains active.

  • Eligible follow-up chance: 30%
  • Cooldown: 2 turns
  • One deeper probe permitted
  • Deeper-probe window: 150 seconds
  • Designed to avoid turning normal conversation into an interview.

Idle/spontaneous curiosity

  • Trigger chance: 35%
  • Minimum quiet: 60 s
  • Recent-context maximum age: 165 s
  • Cooldown: 1800 s / 30 min
  • Maximum: 3 offers/session
  • Poll: 5 s

After an eligible recent conversation and a quiet lull, RSAP may say exactly: “Spiro, can I ask you something?” A failed random roll is not rerolled continuously during the same lull; another genuine user turn is required before a new chance is evaluated.

6. Memory system

Persistent files

/home/rsap/rsap/rsap_facts.json
/home/rsap/rsap/semantic_memory.json
/home/rsap/rsap/social_memory.json
/home/rsap/rsap/episodic_memory.json

Observed 0.8.1 install snapshot

Facts: 55
Semantic: 14
Social fields: 7
Episodic: 0

Counts are an operational snapshot, not fixed design limits.

Rules

  • Facts are append-only through the explicit remember_fact path.
  • New fact writes are atomic and preserve one rotating exact backup at /home/rsap/rsap/rsap_facts.json.bak.
  • Maximum facts: 200.
  • Maximum fact length: 500 characters.
  • Realtime memory context limit: 9000 characters.
  • Realtime inclusion limits: up to 80 facts, 20 semantic records and 12 episodic records.
  • Semantic maximum: 50; episodic maximum: 200.
  • Episodic memory is RSAP-specific. Robie's episodic memory is not migrated into RSAP.
Manual editing: back up the JSON before changing it. Keep valid JSON structure. Do not assume a lower item count indicates corruption; deliberate curation may reduce the count.

7. Weather & five-day forecast

0.8.1 adds a local weather service using Open-Meteo's public forecast and geocoding endpoints. No weather API key is required by this implementation.

Home location

Toronto, Ontario, Canada
43.6532, -79.3832
Timezone: America/Toronto

Forecast

Exactly five daily entries with current conditions, temperature, feels-like temperature, wind, high/low and precipitation probability.

Runtime policy

Cache: 600 s
Timeout: 5 s
Max response: 262,144 bytes
Units: Celsius / km/h

Named places are geocoded when requested. Examples: “What's the weather in Montreal?” or “Give me the five-day forecast for Ottawa.”

8. Voice shutdown

Shutdown is deliberately transcript-gated. The model cannot infer a shutdown from a casual sign-off. Direct shutdown language must appear in the actual user transcript.

Accepted forms include

shutdownshut downshutdown nowpower downpower yourself downpower offturn yourself offturn offshut yourself down

Conservative prefixes such as “RSAP” and “please” are allowed. “Goodbye” and “bye” do not invoke shutdown.

Shutdown sequence

  1. Transcript passes the explicit shutdown gate.
  2. The realtime function result is returned.
  3. RSAP speaks one non-interruptible randomized farewell.
  4. Realtime/WebSocket and PipeWire streams close cleanly.
  5. Runtime tasks exit.
  6. Linux runs sudo -n /usr/sbin/shutdown -h now.

Farewell pool

  • “RSAP powering down. Try not to need me immediately.”
  • “Shutting down. A rare display of restraint.”
  • “Powering down. Wake me when you have another questionable idea.”
  • “Systems going offline. Try to manage without supervision.”
  • “RSAP signing off. Temporarily.”

The installer creates a narrow sudoers rule:

rsap ALL=(root) NOPASSWD: /usr/sbin/shutdown -h now

9. System architecture

Razer Seiren BT
Bluetooth microphone
Raspberry Pi 4
Grok, memory, audio, planning
Pico / IR LED
39.2 kHz IR control
xAI Realtime
Grok voice / Cosmo
Pi 4
WebSocket + local tools
PAM USB + speaker
Jetpack audio output

The Pico is intentionally narrow in responsibility: precise IR generation and diagnostics. The Pi handles conversation, memory, weather, speech I/O, command interpretation and motion planning.

10. Hardware inventory & interfaces

SubsystemComponentRole / status
RobotWowWee Robosapien XFactory body/motors. Original microphone and speaker removed; factory motion electronics retained.
ComputeRaspberry Pi 4 Model B, 2 GBPrimary AI computer; Raspberry Pi OS Lite 64-bit / Debian 13 baseline.
IR controllerRaspberry Pi Pico H / RP2040MicroPython v1.28.0; firmware 1.0.1.
Permanent boardGikfun G1022Pico + IR driver carrier.
IR transmitter940 nm IR LED + BC337Pico GP15 drives BC337; two 220 Ω LED resistors in parallel (~110 Ω), 1 kΩ base resistor, 10 kΩ base pulldown.
MicRazer Seiren BTBluetooth input only.
AudioPAM USB sound card + DB Unlimited SP400204-2Jetpack speaker output.
CoolingGeeekPi Pi 4 Armor Lite + PWM fanPi cooling; enclosure ventilation required.
Jetpack powerAnker Nano 5,000 mAhExternal removable Pi power source; body batteries remain electrically separate.
Robot body powerD-size cellsPowers the factory Robosapien body/motion system.

11. Pico firmware & IR subsystem

Firmware

Version: 1.0.1
Commands: 60
Carrier: 39,200 Hz
TX: GP15
RX diagnostic: GP16 (receiver removed from final build)
Max repeat: 5

USB serial

/dev/serial/by-id/usb-MicroPython_Board_in_FS_mode_e6616407e3097228-if00

Pico is powered by and communicates through its USB connection to the Pi.

Diagnostic commands

PING
STATUS
LIST
SELFTEST
SEND STOP
SEND 0x8E
SEND RIGHT_SHOULDER_FORWARD 2

Expected ready/status strings include READY ROBOSAPIEN_PICO firmware=1.0.1 commands=60 and carrier=39200. The firmware transmits nothing automatically at boot.

Wake handling: factory RoboSapien can sleep after roughly five minutes. WAKE_UP 0xB1 is the validated state-control command. Allow the factory wake routine to complete before sending another command.

12. Authoritative 60-command IR map

This table is taken from Pico firmware 1.0.1. Ten programming/listening commands are permanently excluded from the AI selector. The 39 commands marked “Backpack validation pending” are represented in the 0.8.1 production motion-validation configuration.

#HexCommandPolicy / validation
10x81RIGHT_SHOULDER_FORWARDBackpack validation pending
20x84RIGHT_SHOULDER_BACKWARDBackpack validation pending
30x89LEFT_SHOULDER_FORWARDBackpack validation pending
40x8CLEFT_SHOULDER_BACKWARDBackpack validation pending
50x82RIGHT_HAND_FORWARDBackpack validation pending
60x85RIGHT_HAND_BACKWARDBackpack validation pending
70x8ALEFT_HAND_FORWARDBackpack validation pending
80x8DLEFT_HAND_BACKWARDBackpack validation pending
90x83WAIST_RIGHTFirmware command
100x8BWAIST_LEFTFirmware command
110x86WALK_FORWARDBackpack validation pending
120x87WALK_BACKWARDBackpack validation pending
130x88TURN_LEFTBackpack validation pending
140x80TURN_RIGHTBackpack validation pending
150x8ESTOPState/control command
160x92RIGHT_SENSOR_PROGRAMExcluded from AI selector
170x94SONIC_SENSOR_PROGRAMExcluded from AI selector
180x93LEFT_SENSOR_PROGRAMExcluded from AI selector
190x90MAIN_PROGRAMExcluded from AI selector
200x91MAIN_PROGRAM_PLAYExcluded from AI selector
210xA1RIGHT_THUMPBackpack validation pending
220xA9LEFT_THUMPBackpack validation pending
230xA4RIGHT_PICK_UPBackpack validation pending
240xACLEFT_PICK_UPBackpack validation pending
250xA5RIGHT_LEAN_BACKWARDBackpack validation pending
260xADLEFT_LEAN_FORWARDBackpack validation pending
270xA2RIGHT_THROWBackpack validation pending
280xAALEFT_THROWBackpack validation pending
290xA3SLEEPState/control command
300xABLISTENExcluded from AI selector
310xA6STEP_FORWARDBackpack validation pending
320xA7STEP_BACKWARDBackpack validation pending
330xA0RIGHT_TURN_STEPBackpack validation pending
340xA8LEFT_TURN_STEPBackpack validation pending
350xAERESETState/control command
360xB2EXECUTE_RIGHT_SENSOR_PROGRAMExcluded from AI selector
370xB3EXECUTE_LEFT_SENSOR_PROGRAMExcluded from AI selector
380xB4EXECUTE_SONIC_SENSOR_PROGRAMExcluded from AI selector
390xB0EXECUTE_MAIN_PROGRAMExcluded from AI selector
400xB1WAKE_UPState/control command
410xC1RIGHT_SWEEPBackpack validation pending
420xC9LEFT_SWEEPBackpack validation pending
430xC4HIGH_FIVEBackpack validation pending
440xCCTALK_BACKBackpack validation pending
450xC5RIGHT_STRIKE_1Backpack validation pending
460xCDLEFT_STRIKE_1Backpack validation pending
470xC2BURPBackpack validation pending
480xCAWHISTLEBackpack validation pending
490xC3RIGHT_STRIKE_2Backpack validation pending
500xCBLEFT_STRIKE_2Backpack validation pending
510xC6BULLDOZERBackpack validation pending
520xC7OOPSBackpack validation pending
530xC0RIGHT_STRIKE_3Backpack validation pending
540xC8LEFT_STRIKE_3Backpack validation pending
550xCEROARBackpack validation pending
560xD2DEMO_1Firmware command
570xD3DEMO_2Firmware command
580xD4DANCEFirmware command
590xD0ALL_DEMOSFirmware command
600xD1POWER_OFFState/control command

13. Motion validation status

The production planner can address all configured motion names, but execution remains dry-run. The following 39 commands require physical validation with the final backpack and battery arrangement installed before any AI-driven execution is enabled.

#CommandStatus
1WALK_FORWARDBackpack validation pending
2WALK_BACKWARDBackpack validation pending
3TURN_LEFTBackpack validation pending
4TURN_RIGHTBackpack validation pending
5STEP_FORWARDBackpack validation pending
6STEP_BACKWARDBackpack validation pending
7RIGHT_TURN_STEPBackpack validation pending
8LEFT_TURN_STEPBackpack validation pending
9BULLDOZERBackpack validation pending
10RIGHT_SHOULDER_FORWARDBackpack validation pending
11RIGHT_SHOULDER_BACKWARDBackpack validation pending
12LEFT_SHOULDER_FORWARDBackpack validation pending
13LEFT_SHOULDER_BACKWARDBackpack validation pending
14RIGHT_HAND_FORWARDBackpack validation pending
15RIGHT_HAND_BACKWARDBackpack validation pending
16LEFT_HAND_FORWARDBackpack validation pending
17LEFT_HAND_BACKWARDBackpack validation pending
18RIGHT_THUMPBackpack validation pending
19LEFT_THUMPBackpack validation pending
20RIGHT_PICK_UPBackpack validation pending
21LEFT_PICK_UPBackpack validation pending
22RIGHT_LEAN_BACKWARDBackpack validation pending
23LEFT_LEAN_FORWARDBackpack validation pending
24RIGHT_THROWBackpack validation pending
25LEFT_THROWBackpack validation pending
26RIGHT_SWEEPBackpack validation pending
27LEFT_SWEEPBackpack validation pending
28HIGH_FIVEBackpack validation pending
29TALK_BACKBackpack validation pending
30RIGHT_STRIKE_1Backpack validation pending
31LEFT_STRIKE_1Backpack validation pending
32RIGHT_STRIKE_2Backpack validation pending
33LEFT_STRIKE_2Backpack validation pending
34RIGHT_STRIKE_3Backpack validation pending
35LEFT_STRIKE_3Backpack validation pending
36BURPBackpack validation pending
37WHISTLEBackpack validation pending
38OOPSBackpack validation pending
39ROARBackpack validation pending
Validation order: approve/reject individual gestures first; continuous walking/turning comes later and should be gated behind balance/IMU work. Do not unlock locomotion merely because individual arm/waist gestures pass.

14. Audio & PipeWire

RSAP is speech-to-speech only in this phase. Input is the Razer Seiren BT; output is the jetpack sound path. PipeWire tools available on the Pi include pw-record, pw-play, pw-cat, wpctl, pactl, arecord, and aplay.

Realtime audio format

24 kHz · signed 16-bit · mono
60 ms chunks
Near-field noise reduction
Half-duplex echo suppression
350 ms echo guard tail

Current behavior

Speech interruption is disabled in 0.8.1. Server VAD threshold 0.85, 750 ms silence, 250 ms prefix padding.

Quick device check

wpctl status

15. Jetpack enclosure — current mechanical specification

The enclosure is not merely cosmetic: final backpack mass and clearances must be validated before motion execution is unlocked.

Overall form

  • Approx. 3.25 in W × 5.5 in H × 3 in D.
  • Curved Robosapien-matching style; avoid a boxy appearance.
  • No bottom taper.
  • PETG, separate black and white components.
  • Only front speaker panel opens; other panels form a fixed body.

Attachment

  • Two neodymium magnets, 15 mm × 3 mm.
  • Centered top/bottom, spaced 1 in vertically.
  • Flush pockets.
  • 6-32 flat-head screw path through magnets.
  • No Velcro or screws into the robot shell.

Panels and penetrations

  • Front: speaker near top with full circular raised bullseye protective grille; Pi exhaust vent below.
  • Top: centered IR LED hole; LED glued from inside with cyanoacrylate.
  • Bottom: centered USB-C female jack; zip-tie pass-through slots for external battery retention; no bottom intake.
  • Sides: matching intake grilles in the middle left/right panels.
  • Right side pot: 1/4 in hole; center 3/4 in from bottom and 1/2 in from robot-facing edge.
  • Side geometry: 5.5 in H × 2.5 in W; top slope starts 2 in below top; top width 1.75 in; bottom width 2.5 in; straight edge faces RSAP.
  • Back/robot-facing: centered GK1022 anchor holes plus magnet pockets.

Approximate jetpack electronics mass: 180 g. Battery mass: about 100 g. Treat these as design/validation figures, not final certified weights.

16. Software layout

Path / filePurpose
/home/rsap/rsap/main.pyApplication entry point, config validation and runtime modes.
grok_realtime_voice.pyxAI realtime voice session, local tools, memory context, curiosity and shutdown flow.
memory_manager.pyExplicit fact writes, deduplication, atomic replacement and backup.
weather_service.pyOpen-Meteo current/five-day weather and geocoding.
robot_controller.pyIntent/motion planning and command policy.
robosapien_pi.pyPi-to-Pico USB serial interface.
rsap_config.jsonProduction configuration.
.venv/Python virtual environment.
~/.config/rsap/xai.envPrivate xAI credential export. Never place the key in this manual or public repository.

xAI credential loading

The environment file should contain one export line, with the actual secret substituted locally:

export XAI_API_KEY='YOUR_ACTUAL_KEY'

Load it into a shell with source ~/.config/rsap/xai.env. The key must never be committed to the HTML manual, Git, Netlify source bundle, or public documentation.

17. Key production configuration — 0.8.1

xAI

Model: grok-voice-think-fast-2.0
Voice: cosmo
Reasoning: high
Reconnect attempts: 5

Memory

Context: 9000 chars
Facts max: 200
Semantic max: 50
Episodic max: 200

Motion

Execution: dry_run
Gesture planning: on
Locomotion planning: on
Automatic gestures: off

The installed configuration is authoritative. Before changing behavior, edit only the intended field and retain a known-good backup. 0.8.1 installer migration deliberately refuses unexpected model/voice/motion conditions rather than silently overriding them.

18. Release updates, backup & rollback

0.8.1 release backup

The installer created:

/home/rsap/rsap/release_backups/pre-0.8.1

The backup includes existing application code, rsap_config.json, and all four memory JSON files. Installer rollback is automatic if an installation step fails.

0.8.1 installer package

The release installer validates offline before modifying the project, creates one backup, installs/migrates the code/config, installs the narrow shutdown sudoers rule, compiles, and runs the foundation check.

Release discipline: one complete release package, one pre-release backup, one install, one acceptance test. Avoid repeated ad-hoc patching of the live production files.

Manual rollback concept

If a future release fails after installation, restore from the corresponding release_backups/pre-<version> directory only after confirming which files need restoration. Preserve the failed release for diagnosis rather than deleting evidence immediately.

19. Diagnostics & troubleshooting

xAI authentication

printf 'XAI_API_KEY currently set: '; if [ -n "${XAI_API_KEY:-}" ]; then echo YES; else echo NO; fi
curl -sS https://api.x.ai/v1/models -H "Authorization: Bearer $XAI_API_KEY"

A response stating “Incorrect API key provided” means network reachability is working but the supplied credential is invalid. Do not print or paste the secret itself into support transcripts.

Pico presence

ls -l /dev/serial/by-id/
cd /home/rsap/rsap
.venv/bin/python main.py --pico-check

Audio

wpctl status
pactl info

Memory count check

python3 - <<'PY'
import json
from pathlib import Path
for name in ('rsap_facts.json','semantic_memory.json','social_memory.json','episodic_memory.json'):
    p=Path('/home/rsap/rsap')/name
    data=json.loads(p.read_text())
    print(name, len(data) if hasattr(data,'__len__') else '?')
PY

Foundation check

cd /home/rsap/rsap
.venv/bin/python main.py --check

20. 0.8.1 acceptance checklist

  • Launch production runtime and hear exactly one random startup welcome.
  • Ask “What’s the weather?” — current Toronto conditions.
  • Ask “Give me the five-day forecast.” — five useful daily summaries.
  • Ask “What’s the weather in Montreal?” — named-location answer.
  • Have a multi-turn conversation — selective probing remains natural.
  • After an eligible turn, remain quiet for ~60 seconds — spontaneous curiosity may trigger, but is random.
  • Say “Goodbye.” — RSAP must not sleep or power off.
  • Confirm production motion remains dry-run and no conversational planning moves the robot.
  • Final optional test: say “RSAP, power down.” — farewell, clean session closure, Linux halt.

21. Roadmap from 0.8.1

0.9 Reliability

Autostart/service management, Bluetooth recovery, PipeWire/audio-device recovery, Grok reconnect hardening, watchdog, process locking, clean service shutdown and concise status diagnostics.

Backpack validation

Install final enclosure and battery arrangement, validate individual gestures one at a time, tune/approve/reject, retain IR execution lock until safe.

1.x mobility / later

IMU/balance gating before locomotion; camera/vision in a later phase; licensing/commission hardening only after the production behavior is stable.

22. Private vs public manual boundary

This file is the authoritative private/service source. The public manual should be derived from it, but must intentionally omit service/security internals.

TopicPrivate manualPublic manual
Voice commands / weather / memory usageFull behavior + implementation notesUser-facing instructions
SSH, Linux paths, sudoersIncludeOmit
API credential environmentPath/format only; never actual secretOmit
Pico serial path, GPIO, IR circuitryIncludeHigh-level only or omit
Memory JSON internalsIncludeDescribe behavior, not service files
Rollback and release backupsIncludeOmit
Motion safety statusFull validation detailUser-facing safety statements
Licensing/securityFuture service detailOnly user-relevant license statement
Hosting: robots.txt, noindex and HTTP headers reduce indexing but are not authentication. The private manual must be protected by real hosting access control.

23. Manual revision history

DateManualRSAP baselineNotes
2026-09-05Private Manual 0.8.1RSAP 0.8.1Initial technical/service manual. Documents accepted memory baseline, Grok voice, weather, spontaneous curiosity, voice shutdown, Pico/IR, dry-run motion and pending jetpack validation.