1. Overview
RSAP is a WowWee Robosapien X retrofitted with a standalone Raspberry Pi/Pico AI system. The original robot body retains its native motor electronics and receives control only through infrared commands. The retrofit does not wire the Raspberry Pi directly to the motors.
Identity
Unit: RSAP-0001
Build: Original Prototype
Owner: Spiro Mandylor
Stage: Phase 1
Current capabilities
Realtime speech-to-speech conversation, persistent memory, weather and five-day forecasts, conversational probing, spontaneous curiosity, Pico diagnostics, dry-run motion planning, and voice-triggered Linux shutdown.
Not yet production-enabled
Automatic startup/service recovery, physical AI-driven gesture execution, continuous locomotion, IMU-based motion monitoring, and camera/vision.
2. Safety & release state
0.8.1 safety locks
motion_execution_mode = dry_run- Automatic gestures disabled.
- Continuous locomotion disabled.
- IMU locomotion monitoring disabled.
- Production motion planning does not transmit IR.
- All 39 backpack-dependent motion commands remain physically unvalidated with the final backpack installed.
Hardware handling
- Support RSAP against falls during future movement tests.
- Keep hands, clothing, cables and objects clear of arm/waist travel.
- Do not test continuous walking near stairs or table edges.
- Do not expose the factory
LISTENroutine to AI control; the original mic has been removed.
3. Normal operation
Current manual launch
0.8.1 is not yet the autostart/reliability release. Start the production runtime manually:
ssh rsap@rsap.local
cd /home/rsap/rsap
.venv/bin/python main.py --config /home/rsap/rsap/rsap_config.json --runOn a successful launch, the terminal should show the runtime start, memory counts, Grok session configuration, PipeWire streams, and a production-ready message. RSAP then speaks one randomized startup welcome. The welcome is spoken only once per service launch and should not repeat merely because the realtime connection reconnects.
Startup welcome pool
- “RSAP online. Systems nominal. Your move, Spiro.”
- “Back online. I assume you have a plan.”
- “RSAP online. Try to make this interesting.”
- “Systems are up. What are we working on?”
- “Online again. I was enjoying the silence, but go ahead.”
Stopping the foreground runtime without powering off
From the SSH terminal use Ctrl+C. This stops the foreground Python process. It is different from RSAP's explicit voice shutdown, which halts Linux.
4. Voice commands & natural-language functions
Conversation & utility examples
- “What time is it?”
- “What’s the weather?”
- “Give me the five-day forecast.”
- “What’s the weather in Montreal?”
- “Remember that …”
- “What do you remember about …?”
- Natural conversational questions and follow-ups.
Control examples
- “RSAP, power down.”
- “Please shut down now.”
- “Turn yourself off.”
- “Goodbye” / “bye” are not control commands and do not sleep or shut down RSAP.
Local tool allowlist
| Tool | Purpose |
|---|---|
get_local_time | Current local date/time |
get_system_status | RSAP runtime/system status |
get_pico_status | Pico/IR controller status |
search_memory | Read-only memory lookup |
get_weather | Current + five-day weather |
remember_fact | Explicit append-only fact memory |
request_system_shutdown | Explicit transcript-gated Linux shutdown |
5. AI personality, probing & spontaneous curiosity
RSAP uses xAI Grok Realtime with the Cosmo voice. Personality targets are blunt, calm, competent, candid and dryly humorous rather than sentimental. Current configuration sets humor to 70% and candor to 95%.
Conversational probing
Existing 0.8 behavior remains active.
- Eligible follow-up chance: 30%
- Cooldown: 2 turns
- One deeper probe permitted
- Deeper-probe window: 150 seconds
- Designed to avoid turning normal conversation into an interview.
Idle/spontaneous curiosity
- Trigger chance: 35%
- Minimum quiet: 60 s
- Recent-context maximum age: 165 s
- Cooldown: 1800 s / 30 min
- Maximum: 3 offers/session
- Poll: 5 s
After an eligible recent conversation and a quiet lull, RSAP may say exactly: “Spiro, can I ask you something?” A failed random roll is not rerolled continuously during the same lull; another genuine user turn is required before a new chance is evaluated.
6. Memory system
Persistent files
/home/rsap/rsap/rsap_facts.json/home/rsap/rsap/semantic_memory.json/home/rsap/rsap/social_memory.json/home/rsap/rsap/episodic_memory.json
Observed 0.8.1 install snapshot
Facts: 55
Semantic: 14
Social fields: 7
Episodic: 0
Counts are an operational snapshot, not fixed design limits.
Rules
- Facts are append-only through the explicit
remember_factpath. - New fact writes are atomic and preserve one rotating exact backup at
/home/rsap/rsap/rsap_facts.json.bak. - Maximum facts: 200.
- Maximum fact length: 500 characters.
- Realtime memory context limit: 9000 characters.
- Realtime inclusion limits: up to 80 facts, 20 semantic records and 12 episodic records.
- Semantic maximum: 50; episodic maximum: 200.
- Episodic memory is RSAP-specific. Robie's episodic memory is not migrated into RSAP.
7. Weather & five-day forecast
0.8.1 adds a local weather service using Open-Meteo's public forecast and geocoding endpoints. No weather API key is required by this implementation.
Home location
Toronto, Ontario, Canada
43.6532, -79.3832
Timezone: America/Toronto
Forecast
Exactly five daily entries with current conditions, temperature, feels-like temperature, wind, high/low and precipitation probability.
Runtime policy
Cache: 600 s
Timeout: 5 s
Max response: 262,144 bytes
Units: Celsius / km/h
Named places are geocoded when requested. Examples: “What's the weather in Montreal?” or “Give me the five-day forecast for Ottawa.”
8. Voice shutdown
Shutdown is deliberately transcript-gated. The model cannot infer a shutdown from a casual sign-off. Direct shutdown language must appear in the actual user transcript.
Accepted forms include
Conservative prefixes such as “RSAP” and “please” are allowed. “Goodbye” and “bye” do not invoke shutdown.
Shutdown sequence
- Transcript passes the explicit shutdown gate.
- The realtime function result is returned.
- RSAP speaks one non-interruptible randomized farewell.
- Realtime/WebSocket and PipeWire streams close cleanly.
- Runtime tasks exit.
- Linux runs
sudo -n /usr/sbin/shutdown -h now.
Farewell pool
- “RSAP powering down. Try not to need me immediately.”
- “Shutting down. A rare display of restraint.”
- “Powering down. Wake me when you have another questionable idea.”
- “Systems going offline. Try to manage without supervision.”
- “RSAP signing off. Temporarily.”
The installer creates a narrow sudoers rule:
rsap ALL=(root) NOPASSWD: /usr/sbin/shutdown -h now9. System architecture
Bluetooth microphone
Grok, memory, audio, planning
39.2 kHz IR control
Grok voice / Cosmo
WebSocket + local tools
Jetpack audio output
The Pico is intentionally narrow in responsibility: precise IR generation and diagnostics. The Pi handles conversation, memory, weather, speech I/O, command interpretation and motion planning.
10. Hardware inventory & interfaces
| Subsystem | Component | Role / status |
|---|---|---|
| Robot | WowWee Robosapien X | Factory body/motors. Original microphone and speaker removed; factory motion electronics retained. |
| Compute | Raspberry Pi 4 Model B, 2 GB | Primary AI computer; Raspberry Pi OS Lite 64-bit / Debian 13 baseline. |
| IR controller | Raspberry Pi Pico H / RP2040 | MicroPython v1.28.0; firmware 1.0.1. |
| Permanent board | Gikfun G1022 | Pico + IR driver carrier. |
| IR transmitter | 940 nm IR LED + BC337 | Pico GP15 drives BC337; two 220 Ω LED resistors in parallel (~110 Ω), 1 kΩ base resistor, 10 kΩ base pulldown. |
| Mic | Razer Seiren BT | Bluetooth input only. |
| Audio | PAM USB sound card + DB Unlimited SP400204-2 | Jetpack speaker output. |
| Cooling | GeeekPi Pi 4 Armor Lite + PWM fan | Pi cooling; enclosure ventilation required. |
| Jetpack power | Anker Nano 5,000 mAh | External removable Pi power source; body batteries remain electrically separate. |
| Robot body power | D-size cells | Powers the factory Robosapien body/motion system. |
11. Pico firmware & IR subsystem
Firmware
Version: 1.0.1
Commands: 60
Carrier: 39,200 Hz
TX: GP15
RX diagnostic: GP16 (receiver removed from final build)
Max repeat: 5
USB serial
/dev/serial/by-id/usb-MicroPython_Board_in_FS_mode_e6616407e3097228-if00
Pico is powered by and communicates through its USB connection to the Pi.
Diagnostic commands
PING
STATUS
LIST
SELFTEST
SEND STOP
SEND 0x8E
SEND RIGHT_SHOULDER_FORWARD 2Expected ready/status strings include READY ROBOSAPIEN_PICO firmware=1.0.1 commands=60 and carrier=39200. The firmware transmits nothing automatically at boot.
WAKE_UP 0xB1 is the validated state-control command. Allow the factory wake routine to complete before sending another command.12. Authoritative 60-command IR map
This table is taken from Pico firmware 1.0.1. Ten programming/listening commands are permanently excluded from the AI selector. The 39 commands marked “Backpack validation pending” are represented in the 0.8.1 production motion-validation configuration.
| # | Hex | Command | Policy / validation |
|---|---|---|---|
| 1 | 0x81 | RIGHT_SHOULDER_FORWARD | Backpack validation pending |
| 2 | 0x84 | RIGHT_SHOULDER_BACKWARD | Backpack validation pending |
| 3 | 0x89 | LEFT_SHOULDER_FORWARD | Backpack validation pending |
| 4 | 0x8C | LEFT_SHOULDER_BACKWARD | Backpack validation pending |
| 5 | 0x82 | RIGHT_HAND_FORWARD | Backpack validation pending |
| 6 | 0x85 | RIGHT_HAND_BACKWARD | Backpack validation pending |
| 7 | 0x8A | LEFT_HAND_FORWARD | Backpack validation pending |
| 8 | 0x8D | LEFT_HAND_BACKWARD | Backpack validation pending |
| 9 | 0x83 | WAIST_RIGHT | Firmware command |
| 10 | 0x8B | WAIST_LEFT | Firmware command |
| 11 | 0x86 | WALK_FORWARD | Backpack validation pending |
| 12 | 0x87 | WALK_BACKWARD | Backpack validation pending |
| 13 | 0x88 | TURN_LEFT | Backpack validation pending |
| 14 | 0x80 | TURN_RIGHT | Backpack validation pending |
| 15 | 0x8E | STOP | State/control command |
| 16 | 0x92 | RIGHT_SENSOR_PROGRAM | Excluded from AI selector |
| 17 | 0x94 | SONIC_SENSOR_PROGRAM | Excluded from AI selector |
| 18 | 0x93 | LEFT_SENSOR_PROGRAM | Excluded from AI selector |
| 19 | 0x90 | MAIN_PROGRAM | Excluded from AI selector |
| 20 | 0x91 | MAIN_PROGRAM_PLAY | Excluded from AI selector |
| 21 | 0xA1 | RIGHT_THUMP | Backpack validation pending |
| 22 | 0xA9 | LEFT_THUMP | Backpack validation pending |
| 23 | 0xA4 | RIGHT_PICK_UP | Backpack validation pending |
| 24 | 0xAC | LEFT_PICK_UP | Backpack validation pending |
| 25 | 0xA5 | RIGHT_LEAN_BACKWARD | Backpack validation pending |
| 26 | 0xAD | LEFT_LEAN_FORWARD | Backpack validation pending |
| 27 | 0xA2 | RIGHT_THROW | Backpack validation pending |
| 28 | 0xAA | LEFT_THROW | Backpack validation pending |
| 29 | 0xA3 | SLEEP | State/control command |
| 30 | 0xAB | LISTEN | Excluded from AI selector |
| 31 | 0xA6 | STEP_FORWARD | Backpack validation pending |
| 32 | 0xA7 | STEP_BACKWARD | Backpack validation pending |
| 33 | 0xA0 | RIGHT_TURN_STEP | Backpack validation pending |
| 34 | 0xA8 | LEFT_TURN_STEP | Backpack validation pending |
| 35 | 0xAE | RESET | State/control command |
| 36 | 0xB2 | EXECUTE_RIGHT_SENSOR_PROGRAM | Excluded from AI selector |
| 37 | 0xB3 | EXECUTE_LEFT_SENSOR_PROGRAM | Excluded from AI selector |
| 38 | 0xB4 | EXECUTE_SONIC_SENSOR_PROGRAM | Excluded from AI selector |
| 39 | 0xB0 | EXECUTE_MAIN_PROGRAM | Excluded from AI selector |
| 40 | 0xB1 | WAKE_UP | State/control command |
| 41 | 0xC1 | RIGHT_SWEEP | Backpack validation pending |
| 42 | 0xC9 | LEFT_SWEEP | Backpack validation pending |
| 43 | 0xC4 | HIGH_FIVE | Backpack validation pending |
| 44 | 0xCC | TALK_BACK | Backpack validation pending |
| 45 | 0xC5 | RIGHT_STRIKE_1 | Backpack validation pending |
| 46 | 0xCD | LEFT_STRIKE_1 | Backpack validation pending |
| 47 | 0xC2 | BURP | Backpack validation pending |
| 48 | 0xCA | WHISTLE | Backpack validation pending |
| 49 | 0xC3 | RIGHT_STRIKE_2 | Backpack validation pending |
| 50 | 0xCB | LEFT_STRIKE_2 | Backpack validation pending |
| 51 | 0xC6 | BULLDOZER | Backpack validation pending |
| 52 | 0xC7 | OOPS | Backpack validation pending |
| 53 | 0xC0 | RIGHT_STRIKE_3 | Backpack validation pending |
| 54 | 0xC8 | LEFT_STRIKE_3 | Backpack validation pending |
| 55 | 0xCE | ROAR | Backpack validation pending |
| 56 | 0xD2 | DEMO_1 | Firmware command |
| 57 | 0xD3 | DEMO_2 | Firmware command |
| 58 | 0xD4 | DANCE | Firmware command |
| 59 | 0xD0 | ALL_DEMOS | Firmware command |
| 60 | 0xD1 | POWER_OFF | State/control command |
13. Motion validation status
The production planner can address all configured motion names, but execution remains dry-run. The following 39 commands require physical validation with the final backpack and battery arrangement installed before any AI-driven execution is enabled.
| # | Command | Status |
|---|---|---|
| 1 | WALK_FORWARD | Backpack validation pending |
| 2 | WALK_BACKWARD | Backpack validation pending |
| 3 | TURN_LEFT | Backpack validation pending |
| 4 | TURN_RIGHT | Backpack validation pending |
| 5 | STEP_FORWARD | Backpack validation pending |
| 6 | STEP_BACKWARD | Backpack validation pending |
| 7 | RIGHT_TURN_STEP | Backpack validation pending |
| 8 | LEFT_TURN_STEP | Backpack validation pending |
| 9 | BULLDOZER | Backpack validation pending |
| 10 | RIGHT_SHOULDER_FORWARD | Backpack validation pending |
| 11 | RIGHT_SHOULDER_BACKWARD | Backpack validation pending |
| 12 | LEFT_SHOULDER_FORWARD | Backpack validation pending |
| 13 | LEFT_SHOULDER_BACKWARD | Backpack validation pending |
| 14 | RIGHT_HAND_FORWARD | Backpack validation pending |
| 15 | RIGHT_HAND_BACKWARD | Backpack validation pending |
| 16 | LEFT_HAND_FORWARD | Backpack validation pending |
| 17 | LEFT_HAND_BACKWARD | Backpack validation pending |
| 18 | RIGHT_THUMP | Backpack validation pending |
| 19 | LEFT_THUMP | Backpack validation pending |
| 20 | RIGHT_PICK_UP | Backpack validation pending |
| 21 | LEFT_PICK_UP | Backpack validation pending |
| 22 | RIGHT_LEAN_BACKWARD | Backpack validation pending |
| 23 | LEFT_LEAN_FORWARD | Backpack validation pending |
| 24 | RIGHT_THROW | Backpack validation pending |
| 25 | LEFT_THROW | Backpack validation pending |
| 26 | RIGHT_SWEEP | Backpack validation pending |
| 27 | LEFT_SWEEP | Backpack validation pending |
| 28 | HIGH_FIVE | Backpack validation pending |
| 29 | TALK_BACK | Backpack validation pending |
| 30 | RIGHT_STRIKE_1 | Backpack validation pending |
| 31 | LEFT_STRIKE_1 | Backpack validation pending |
| 32 | RIGHT_STRIKE_2 | Backpack validation pending |
| 33 | LEFT_STRIKE_2 | Backpack validation pending |
| 34 | RIGHT_STRIKE_3 | Backpack validation pending |
| 35 | LEFT_STRIKE_3 | Backpack validation pending |
| 36 | BURP | Backpack validation pending |
| 37 | WHISTLE | Backpack validation pending |
| 38 | OOPS | Backpack validation pending |
| 39 | ROAR | Backpack validation pending |
14. Audio & PipeWire
RSAP is speech-to-speech only in this phase. Input is the Razer Seiren BT; output is the jetpack sound path. PipeWire tools available on the Pi include pw-record, pw-play, pw-cat, wpctl, pactl, arecord, and aplay.
Realtime audio format
24 kHz · signed 16-bit · mono
60 ms chunks
Near-field noise reduction
Half-duplex echo suppression
350 ms echo guard tail
Current behavior
Speech interruption is disabled in 0.8.1. Server VAD threshold 0.85, 750 ms silence, 250 ms prefix padding.
Quick device check
wpctl status15. Jetpack enclosure — current mechanical specification
The enclosure is not merely cosmetic: final backpack mass and clearances must be validated before motion execution is unlocked.
Overall form
- Approx. 3.25 in W × 5.5 in H × 3 in D.
- Curved Robosapien-matching style; avoid a boxy appearance.
- No bottom taper.
- PETG, separate black and white components.
- Only front speaker panel opens; other panels form a fixed body.
Attachment
- Two neodymium magnets, 15 mm × 3 mm.
- Centered top/bottom, spaced 1 in vertically.
- Flush pockets.
- 6-32 flat-head screw path through magnets.
- No Velcro or screws into the robot shell.
Panels and penetrations
- Front: speaker near top with full circular raised bullseye protective grille; Pi exhaust vent below.
- Top: centered IR LED hole; LED glued from inside with cyanoacrylate.
- Bottom: centered USB-C female jack; zip-tie pass-through slots for external battery retention; no bottom intake.
- Sides: matching intake grilles in the middle left/right panels.
- Right side pot: 1/4 in hole; center 3/4 in from bottom and 1/2 in from robot-facing edge.
- Side geometry: 5.5 in H × 2.5 in W; top slope starts 2 in below top; top width 1.75 in; bottom width 2.5 in; straight edge faces RSAP.
- Back/robot-facing: centered GK1022 anchor holes plus magnet pockets.
Approximate jetpack electronics mass: 180 g. Battery mass: about 100 g. Treat these as design/validation figures, not final certified weights.
16. Software layout
| Path / file | Purpose |
|---|---|
/home/rsap/rsap/main.py | Application entry point, config validation and runtime modes. |
grok_realtime_voice.py | xAI realtime voice session, local tools, memory context, curiosity and shutdown flow. |
memory_manager.py | Explicit fact writes, deduplication, atomic replacement and backup. |
weather_service.py | Open-Meteo current/five-day weather and geocoding. |
robot_controller.py | Intent/motion planning and command policy. |
robosapien_pi.py | Pi-to-Pico USB serial interface. |
rsap_config.json | Production configuration. |
.venv/ | Python virtual environment. |
~/.config/rsap/xai.env | Private xAI credential export. Never place the key in this manual or public repository. |
xAI credential loading
The environment file should contain one export line, with the actual secret substituted locally:
export XAI_API_KEY='YOUR_ACTUAL_KEY'Load it into a shell with source ~/.config/rsap/xai.env. The key must never be committed to the HTML manual, Git, Netlify source bundle, or public documentation.
17. Key production configuration — 0.8.1
xAI
Model: grok-voice-think-fast-2.0
Voice: cosmo
Reasoning: high
Reconnect attempts: 5
Memory
Context: 9000 chars
Facts max: 200
Semantic max: 50
Episodic max: 200
Motion
Execution: dry_run
Gesture planning: on
Locomotion planning: on
Automatic gestures: off
The installed configuration is authoritative. Before changing behavior, edit only the intended field and retain a known-good backup. 0.8.1 installer migration deliberately refuses unexpected model/voice/motion conditions rather than silently overriding them.
18. Release updates, backup & rollback
0.8.1 release backup
The installer created:
/home/rsap/rsap/release_backups/pre-0.8.1The backup includes existing application code, rsap_config.json, and all four memory JSON files. Installer rollback is automatic if an installation step fails.
0.8.1 installer package
The release installer validates offline before modifying the project, creates one backup, installs/migrates the code/config, installs the narrow shutdown sudoers rule, compiles, and runs the foundation check.
Manual rollback concept
If a future release fails after installation, restore from the corresponding release_backups/pre-<version> directory only after confirming which files need restoration. Preserve the failed release for diagnosis rather than deleting evidence immediately.
19. Diagnostics & troubleshooting
xAI authentication
printf 'XAI_API_KEY currently set: '; if [ -n "${XAI_API_KEY:-}" ]; then echo YES; else echo NO; fi
curl -sS https://api.x.ai/v1/models -H "Authorization: Bearer $XAI_API_KEY"A response stating “Incorrect API key provided” means network reachability is working but the supplied credential is invalid. Do not print or paste the secret itself into support transcripts.
Pico presence
ls -l /dev/serial/by-id/
cd /home/rsap/rsap
.venv/bin/python main.py --pico-checkAudio
wpctl status
pactl infoMemory count check
python3 - <<'PY'
import json
from pathlib import Path
for name in ('rsap_facts.json','semantic_memory.json','social_memory.json','episodic_memory.json'):
p=Path('/home/rsap/rsap')/name
data=json.loads(p.read_text())
print(name, len(data) if hasattr(data,'__len__') else '?')
PYFoundation check
cd /home/rsap/rsap
.venv/bin/python main.py --check20. 0.8.1 acceptance checklist
- Launch production runtime and hear exactly one random startup welcome.
- Ask “What’s the weather?” — current Toronto conditions.
- Ask “Give me the five-day forecast.” — five useful daily summaries.
- Ask “What’s the weather in Montreal?” — named-location answer.
- Have a multi-turn conversation — selective probing remains natural.
- After an eligible turn, remain quiet for ~60 seconds — spontaneous curiosity may trigger, but is random.
- Say “Goodbye.” — RSAP must not sleep or power off.
- Confirm production motion remains dry-run and no conversational planning moves the robot.
- Final optional test: say “RSAP, power down.” — farewell, clean session closure, Linux halt.
21. Roadmap from 0.8.1
0.9 Reliability
Autostart/service management, Bluetooth recovery, PipeWire/audio-device recovery, Grok reconnect hardening, watchdog, process locking, clean service shutdown and concise status diagnostics.
Backpack validation
Install final enclosure and battery arrangement, validate individual gestures one at a time, tune/approve/reject, retain IR execution lock until safe.
1.x mobility / later
IMU/balance gating before locomotion; camera/vision in a later phase; licensing/commission hardening only after the production behavior is stable.
22. Private vs public manual boundary
This file is the authoritative private/service source. The public manual should be derived from it, but must intentionally omit service/security internals.
| Topic | Private manual | Public manual |
|---|---|---|
| Voice commands / weather / memory usage | Full behavior + implementation notes | User-facing instructions |
| SSH, Linux paths, sudoers | Include | Omit |
| API credential environment | Path/format only; never actual secret | Omit |
| Pico serial path, GPIO, IR circuitry | Include | High-level only or omit |
| Memory JSON internals | Include | Describe behavior, not service files |
| Rollback and release backups | Include | Omit |
| Motion safety status | Full validation detail | User-facing safety statements |
| Licensing/security | Future service detail | Only user-relevant license statement |
robots.txt, noindex and HTTP headers reduce indexing but are not authentication. The private manual must be protected by real hosting access control.23. Manual revision history
| Date | Manual | RSAP baseline | Notes |
|---|---|---|---|
| 2026-09-05 | Private Manual 0.8.1 | RSAP 0.8.1 | Initial technical/service manual. Documents accepted memory baseline, Grok voice, weather, spontaneous curiosity, voice shutdown, Pico/IR, dry-run motion and pending jetpack validation. |