By: Robert Walsh
Have you ever played a video game where all you had to do was tilt the controller to direct the on-screen action? The Nintendo Wii was one of the first gaming systems to employ this technology when it was released in 20061, and smartphone and tablet games like Temple Run and Subway Surfers use it too. Did you stop playing the game long enough to wonder how this works?
In this lesson, we will use a Raspberry Pi with the Sense HAT module to create a simple scene where you just need to tilt the Raspberry Pi to control a rocket ship. To complete this activity, you will need:
- A Raspberry Pi 3 or 4 (2 GB RAM recommended)
- A Raspberry Pi Sense HAT module
- Scratch 3 with the Sense HAT extensions
- Note: Scratch 3 is preinstalled with the full version of the Raspberry Pi OS
The video below will walk you through creating a Scratch program that allows you to control a rocket ship in space using a Raspberry Pi with a Sense HAT module. This is the same technology used in smartphones and video game controllers.
If you need help getting your program working, try these downloads:
Note: The complete project download is a ZIP file and will have to be expanded to get to the Scratch program. Scratch programs have an .SB3 file extension.
The Sense HAT contains an inertial monitoring unit (IMU) that is able to detect motion2. One component of the IMU is a gyroscope that detects rotation. When the Sense HAT is tilted forward or backward, a value called roll changes. When it is tilted left or right, the pitch changes. The Sense HAT can also detect a twisting rotation (like if you left the Raspberry Pi flat on a surface and rotated it to the left or right without tilting it). This is called yaw. Our program did not use the yaw value.
Airplanes, spacecraft, and other flying objects use pitch, roll, and yaw to describe their motion3. Pitch generally indicates whether the object is climbing or diving, while roll typically the tilt to the left or the right. (The Sense HAT extension for Scratch, though, uses forward and backward for roll and left or right for pitch.). Yaw shows where the nose is pointed.
This project also taught us about the XY coordinate plane. We used XY coordinates to position our sprite and to indicate which of the Sense HAT’s LEDs we wanted lit. For the sprite, the origin (the center or the point whose X and Y coordinates are both zero) was located in the center of the stage. X values greater than zero were to the right of center, while those with values less than zero were to the left. Y values greater than zero were above the center, and those less than zero were below. To move the rocket to the right, we increased the X coordinate, and to move to the left, we decreased the X. Likewise, to go up, we increased the Y, and to go down, we decreased it.
The Sense HAT LED matrix uses XY coordinates, too, but the origin is not in the center. It is in the upper left corner. As we move to the right, the X coordinate gets larger. However, as we move down, the Y coordinate gets larger. This is the opposite of the coordinate system for the sprite where larger Y values were higher on the screen. Another important point about the coordinates for the LEDs on the Sense HAT is that the numbering starts at zero, not one. For example, an LED in the first column has an X coordinate of 0 and one in the last column has an X coordinate of 7 even though there are 8 columns. The same is true for the Y coordinates, but instead of columns, the Y indicates the row.
In addition to game controllers and flying crafts, gyroscopes and IMUs are used in other devices, too. Segways, innovative personal transportation devices sometimes used by police officers and security guards in places where other vehicles are impractical, also use internal gyroscopes to help keep the Segway upright and to detect which way the rider is leaning4.
There are many ways that we could extend the rocket ship program. For example, we could make the ship move faster in a given direction based on how long the Sense HAT is tilted. This would make the movement more realistic. We could also use the yaw value to turn the ship rather than the left or right tilt. This might be a more appropriate control system game with a car or even a person. A game viewed in the first-person where the player sees what the character sees could use both tilt and yaw. One could turn the character, while the other could just turn the head to look in a different direction.
Another change could involve adding obstacles that the player must avoid. Or the player could be expected to collect the other objects on the screen rather than to avoid them.
What other adaptations could you think to make?
What we learned in this lesson:
- How to interact with the Sense HAT module using Scratch 3 on a Raspberry Pi
- How the gyroscope component of the IMU detects motion and reports pitch and roll
- How to locate an object on an XY coordinate plane
How did you do with the lesson?
- What parts were easy and what parts were confusing?
- Were any parts a review of things you already knew?
- What would you like to know more about?
1Romero, J. (2006, December 18). How do motion-sensing video game controllers work? ScienceLine. https://scienceline.org/2006/12/motioncontrollers/
2Raspberry Pi Foundation. (n.d.). Getting started with the Sense HAT. https://projects.raspberrypi.org/en/projects/getting-started-with-the-sense-hat/8
3SoftwarePole. (2015, March 25). Airplane control – roll, pitch, yaw [YouTube video]. https://www.youtube.com/watch?v=pQ24NtnaLl8
4Harris, T. (n.d.). How Segways work. How Stuff Works. https://science.howstuffworks.com/transport/engines-equipment/ginger.htm