This tutorial is based on a workshop presented
at HRI summer school on social robots organized by:
- Vrije Universiteit Brussel – Robotics &
Multibody Mechanics Research Group: Ir Cao Hoang Long, Prof dr ir Bram
Vanderborght and Ir Greet Van de Perre
- Plymouth University: Prof dr ir Tony Belpaeme
and James Kennedy
My Keepon, a simple and low-cost version of
Keepon robot and the commercial robot, is a black box with two buttons. For
human-robot interaction studies, it needs to be controlled by a PC. The goal of
this tutorial is to hack the robot so the movements and sounds of My Keepon
robot can be controlled independently over a PC Serial communication (UART).
This tutorial is based on the work of:
Required parts and software
The costs for hacking Keepon is around 50€
depending on the configuration.
Download and install the following programs on
your laptop. Run everything as administrator of the PC,
otherwise some components may not work.
1) Download and install Arduino. A manual how to
do so, can be found on: http://arduino.cc/en/Guide/HomePage
2) Download the source code of BeatBots
firmware for the Arduino: MyKeepon.ino
3) Download and install MS Visual C# 2010 Express:
4) Download ViKeepon software: keeponhacking.zip
*** Another possibility is to use a hacked
firmware which we can send raw commands to My Keepon (see Appendix)
2) Firmware (Arduino 0023 – Baud rate 9600): keepon_i2c_comm_raw.pde
4) Software: ViKeepon-original.zip
*** If you don’t need to know how My Keepon is
hacked, skip 3-Hacking principle and go to 4.
My Keepon uses two microprocessors to control
the movements and the sounds, which talk to each other via I2C protocol.
- The PS232 (address 0x52) deals with sounds and
encoders and is the slave node on the I2C bus.
- The PS234 (address 0x55) handles driving the H-bridges, main processing (including handling button presses), and is the master node on the I2C bus. At the bottom-right corner of the PCB , there are 4 pads (V, CI, DA and G) of I2C communication. Those pads are connected to A0, A5, A4 and GND of Arduino Uno respectively with a logic voltage converter (5V to 3.3V).
In order to control the My Keepon, the master
microprocessor must be changed to be a slave. This step is done by pulling the
two I2C lines to the ground for one second after the robot powers up. (For the
powering up detection, the V line is connected to Analog pin A0 of Arduino).
Therefore it is important first to power up the Arduino and then the robot as
will be stated in the start-up procedure.
In order to connect the I2C connection with
the USB port of the PC an Arduino will be used with dedicated shield as shown
in Fig 2. The shield has simple electronics to do the logic conversion between
5V and 3.3V.
STEP 1: Opening My Keepon
To open My Keepon, use a screwdriver for the 4
screws in the back casing of My Keepon. Be careful when opening My Keepon, take
care of the position of the different parts, so you are able to reassemble them
again later. With a drill make a hole
above the power connector in order to let the four wires enter the casing.
How to open My Keepon casing.
STEP 2: Connect My Keepon hardware to Arduino
Attach with some tape the four wires along the
black flat cable. Under a transparent plastic cover is the PCB of My Keepon
(located in the red rounded rectangle). Solder the four wires in the correct
order, solder the other ends to the shield. Check if V of My Keepon is
connected with V of Arduino shield, same for CI, DA and G.
Where to solder the wires
STEP 3: Re-assembly
Re-assemble the robot carefully and tighten
the back cover with the 4 screws. Power up My Keepon. Without power for Arduino,
My Keepon should work as it did at purchase. So My Keepon can still be used as
you did when it was purchased.
STEP 4: Connect the hardware to your computer
Plug the Arduino shield to Arduino Uno with
the corresponding pins. Then, connect the board to your computer by a USB
STEP 5: Install the firmware
Open the Arduino IDE and upload the software
with .ino extension in the Arduino. This should only be done once. In Control
Panel under Devices check the port number to which the Arduino is connected.
Normally My Keepon is now hacked and ready for
its first use.
How to use the hacked My Keepon
Important is always to follow this bullet
- 1) Connect the Arduino to My Keepon.
- 2) Connect the Arduino to a PC using a USB cable.
- 3) Launch ViKeepon.exe
- 4) Power up My Keepon.
(It is important to first power Arduino (by
the USB cable) and then power up My Keepon as such the Arduino takes over the
control. Normally My Keepon should not
move. When you turn on My Keepon, if it starts moving the Arduino has no
control. Then, you need to redo the 4 steps in the bullet list.)
When opening the .exe file of ViKeepon, the
user interface will open.
Select the correct port (as found in Control
Press the button “Open Port”.
ViKeepon User interface
Press the button “Jump”. If My Keepon jumps,
jump as well of joy. You just hacked My Keepon and you have control over him by
the ViKeepon User Interface.
The different items of the user interface are
as follows. Try the different possibilities to get familiar with the different
possibilities of My Keepon.
- Settings: Configuration of serial port: Port
number, Baud Rate, Parity, Data Bits, Stop bits and Close/Open button.
- Sending and Receiving data window: Send the
arbitrary command and display the outgoing and incoming data (commands) – only
- Movement: 2 buttons and 4 numeric track-bars
(with for two up-down boxes to change the movements of My Keepon (base pan,
tilt, 5 position lean, 4 position height).
- Mode buttons: start the Dance, Touch, Tempo,
Sleep mode and stop button (although functioning is not sure).
- Sounds: 15 buttons corresponding to 15
different sounds generated by My Keepon’s hardware.
The program can also read the status of the
buttons in the body, responded data is displayed in the data window.
With the Visual Studio programming environment
you are able to modify the ViKeepon software. Make a copy of the software so
you have a backup and start modifying the software.
In order to modify the software, the basics of
the communication protocol must be understood.
UART protocol is used to communicate between
computer software and Arduino. Arduino receives data from My Keepon (button
states and motor positions) and computer software sends commands to control the
movements and sounds of My Keepon. (COM port settings: baud rate 115200, parity
none, data bits 8, stop bits 1).
Basically My Keepon reacts to commands that
are sent to and received from the robot over the UART protocol using hex
values. Persons interested in this can see in appendix the communication
protocol. We have a program that uses these commands, persons interested in
this can contact the organizers of this workshop.
To make these commands more understandable,
the following commands have been offered by BeatBots.
Turn on your My Keepon. You should receive a "My Keepon
Allowable commands (the closing semicolon is
SOUND PLAY <0...63>;
SOUND REPEAT <0...63>;
SOUND DELAY <msec>;
SPEED [PAN, TILT, PONSIDE] <0...255>;
SIDE [CYCLE, CENTERFROMLEFT, RIGHT, CENTERFROMRIGHT, LEFT];
PON [UP, HALFDOWN, DOWN, HALFUP];
Strings that the Arduino can send back to you:
BUTTON [DANCE, TOUCH] [OFF, ON]
BUTTON [HEAD, FRONT, BACK, RIGHT, LEFT] [OFF, ON]
MOTOR [PAN, TILT, SIDE, PON] FINISHED
MOTOR [PAN, TILT, SIDE, PON] STALLED
ENCODER TILT [NOREACH, FORWARD, BACK, UP]
ENCODER PON [HALFDOWN, UP, DOWN, HALFUP]
ENCODER SIDE [CENTER, RIGHT, LEFT]
ENCODER PAN [BACK, RIGHT, LEFT, CENTER]
[PAN, TILT, PONSIDE] [-127...127]
POSITION [PAN, TILT, PONSIDE] [VAL]
AUDIO TEMPO [67, 80, 100, 133, 200] (if BPM cannot be detected, this is
estimated from power spectral density response)
AUDIO MEAN [0...64] (the mean of the envelope over a 1.28sec window, max
around 64 for very loud music, not updated when motors are moving)
AUDIO RANGE [0...64] (dynamic range, max 64 for shouting, not updated
when motors moving)
AUDIO ENVELOPE [0...127] (near instantaneous log of the audio amplitude;
commented out in code for reduction of data transfer)
AUDIO BPM [VAL] (estimated beat interval in multiples of 5msec)
The easiest would be to see the code for the
different buttons and compare it with the command structure given above.
The first exercise is to make a motion and
sound sequence for the button “Harlem Shake”. Search for the string “//ADD
HARLEM SHAKE SEQUENCE HERE”. You can add here your code using the allowable
commands for the robot.
The second exercise is to modify the reaction
of My Keepon when one of the robot’s buttons is pressed.
Search for this string in the software code:
“//ADD HERE CODE TO MODIFY MOTION AND SOUNDS OF BUTTON TOP OF HEAD”.
The third exercise is now up to you! We are
curious which behaviours you will give to My Keepon. Good luck!!!
Data package structure
4th byte to (n+3)th byte
Length of data (n)
- Device Address:
- 0x52 - PS232: Sounds controller
- 0x55 - PS234: Movement controller
- 0x50: Button states
- Length of data: numbers of bytes that will be
written/read to/from the device
- Write mode: Device index, Motor position,
Movement speed... (see in the I2C protocol commands)
- Read mode: No data
I2C Protocol Commands
The commands of I2C protocol is used to communicate
between Arduino and hardware of My Keepon (Thanks to the research of
- Device addresses:
- 0x52 - Sound bank access/playing
- 0x55 - Motors for turning whole unit
- 0x50 – Buttons
- Device Sleep: Assuming no interaction takes
place for 3 minutes, the robot will put itself to sleep. This will cause the
data/clock lines on the I2C bus to both drops to ground.
- Device Wakeup: To wake up the device from
sleep mode, pull the V line on the exposed I2C pads to ground.
Motor control command structure
This command is used to control all movements
of My Keepon: base rotation, forward/backward bending, left/right bending and
- Motor index:
- Base rotation: 0x04
- Forward/Backward Bend: 0x02
- Bobbing and Left/Right Bending: 0x00
- Motor position:
- Base rotation: 0x00 – 0xFF
- Forward/Backward Bend: 0x00 – 0xFF
- Bobbing: 0x80 – 0xFF
- Left/Right Bending: 0x00 – 0x7F
Motor macro structure
There are available programs (macros) for
specific actions in My Keepon. We are able to run those macros by using motor
- Macro: 0x20, 0x21, 0x22, 0x23, 0x24, 0xA0,
0xA1, 0xA2 and 0xA3
Status about the position of motors is
retrieved by requesting 13 bytes from the 0x55 device. Due to the lack of
information from the manufacturer, we only know the meaning of the last two
- Byte 0x0B - Front/Back Encoder
- Byte 0x0C - Base Rotation Encoder
- Sound index:
- 0x91 - Up Wakeup Sound
- 0x92 - Down Wakeup Sound
- 0x95 - Yawn Down
- 0x96 - Initial Boot Sound?
- 0x97 - Sigh
- 0x9A - Yawn Up
- 0x9B - Sleep (Sound played before turning off)
- 0x9C - Chirp
- 0xAC - Whine
- 0xBC - Beep noise when head hit
- 0xBD - Up/Down noise when squatting
- 0xBF - Sneeze Up
- 0xC0 - Sneeze Down
Button status is found from reading a single
byte from Device 0x50.
- Button Index:
- 0x01 - Music Mode Front Button
- 0x02 - Unused
- 0x04 - Top of Head
- 0x08 - Touch Mode Front Button
- 0x10 - Left Body (Facing toward user)
- 0x20 - Front Body
- 0x40 - Right Body (Facing toward user)
- 0x80 - Back Body