Individual Programming Assignment 4

From CS 160 Fall 2010

Jump to: navigation, search

Contents

Overview

The goal of this assignment is to use an external sensor (a Wii Remote) to gather a stream of data, and to visualize it in real time. By placing visual markers on various parts of the body, it affords direct input from movement without a mediating input device. You need to use an actual bluetooth-enabled device (there is no bluetooth emulation in the emulator), and you need to use a real Wii remote for testing, so all the development for this project will be on an Archos 5 device, or on your own Android device.

The core technologies are (i) Bluetooth and (ii) multithreaded programming. You only interact with these at very high level, so you wont need to write much code. You will also be making use of (iii) Android's native code interface, although that part of the assignment has already been built for you. But it is useful to understand that part of the system, so a link is included for you can understand it better and try building the native library for yourself. Before actually getting started on the assignment, we review some of these concepts:

Background on Bluetooth

Android's Native Code Interface

Background on the Wii Remote API

Getting Started

Download the startup project for this assignment now. Make sure you have set up communication with your Archos device, as per the "Tips and Tricks" section on the left. Try building the project, and installing on your device. Make sure Bluetooth is turned on in the Device's settings. The App name is "runBT".

When the app starts you will see a single button that says "Find Wiimotes". To put a Wii Remote in discovery mode, press buttons 1 and 2 at the same time. The LEDs should flash (the number that flash is proportional to battery capacity). Click the "Find Wiimotes" button on your Archos. After a few seconds, the Wii Remote should rumble and LED 1 should come on steadily, and the Archos display should say "Found 1 wiimotes".

Getting IR events

To start, you need a source of infrared light. You can use the normal Wii "sensor bar" which has two points, or an incandescent (not LED) flashlight, or an IR LED lamp, if you have one lying around. For flashlights, the source needs to be "small" in the Wii Remote's field of view. Small flashlights (like MagLites) work well at a foot or two. A large flashlight (1" or more reflector) would need to be further away. If you use the Wii sensor bar, make sure the Wii is off when you try to connect the Wii Remote to your Archos. Otherwise it will try to connect to the Wii instead.

The Wiimote can generate a lot of IR events, and these are not automatically sent to an application. To enable them you have to call activateIRTRacking() (watch capitalization in the method name) on the wiimote object. To do something with wiimote events, you need to instantiate a new WiimoteListener. This is an interface, so you need to define all of its methods. The only one that needs to do anything is onIrEvent(IREvent e). The IREvent includes a method for getting the visible IR points, which is getIRPoints(). It returns a variable-length array of points of type IRSource. The length of the array equals the number of points that the Wii Remote detects, which is between zero and four. Finally an IRSource has X and Y point fields, and you want the "raw" version, which you can access with getRx() and getRy(). So now you have the X and Y coordinates of up to 4 points that the Wii Remote can "see".

Drawing Points

Drawing the points found by the Wiimote is fairly straightforward. Just like IPA2, you create a custom View class and override its onDraw(Canvas C) method. Unless you want to draw trails, you will clear the canvas each time, and then paint the number of points that were received from the last IREvent.

You can trigger the onDraw() method by calling invalidate() on the Canvas. The only caveat is that you can't do this directly inside your onIrEvent code. WiiuseJ has its own thread which is monitoring native code events and interpreting those as events by calling the event handlers like onIrEvent. This other thread isn't allowed to "touch" any of the GUI components, e.g. by calling invalidate() on them. And that wouldn't be good design anyway. Instead, your onIrEvent method should save the found IR points somewhere, and then pass something to a Handler associated with the main (GUI) thread. This "something" could be a message, since this is the most common event from the WiiuseJ thread that this particular program needs to deal with. If its a message, you will need to save the IRpoints in a variable in the main (Activity) class, and override the handleMessage() method (which actually calls invalidate()) when you create the GUI thread's Handler. You could also pass a runnable to the GUI thread's Handler. This runnable would then contain the IRSource points in its own instance variables, and could implement drawing plus invalidate() in a run() method.

Keeping the GUI Alive

The starter project has one serious weakness which is that it calls getWiimotes() from the GUI thread. Since this routine takes several seconds (and may get wedged), the GUI loses liveness. During this time it is not clear what is happening either, because it is not possible to update the display (e.g. changing a button or textView to show that the app is searching for Wiimotes) until the onClick handler for the button returns. Updates to the display can only happen on return from GUI event handling code. Since getWiimotes runs for several seconds, no updates to the display happen until the system has found (or failed to find) a Wii Remote.

A better solution is to hand off the search for Wii Remotes to a worker thread. That is, the onClick handler for the "find Wiimotes" button should start a worker task that actually checks for Wiimotes. It should immediately update the button text to indicate that it is searching. And when the search finishes, it should update the textView next to the button with the number of Wiimotes found. That implies that the worker task should post a message or runnable back to the GUI thread when it finishes.

Once you have found a wiimote, the "Find Wiimotes" button should display "disconnect" to allow users to disconnect the wiimote. The onClick handler for this button should perform disconnection if the wiimote is already connected when it is pressed.

Keeping the Screen Alive

The Archos includes a Powersaving screen dimmer. If you dont interact with the GUI after a few minutes it will blank the display, requiring you to press the "start" button on the top of the device to get the display back. This process plays havoc with the BT subsystem, and stops the device from streaming sensor data from the Wiimote. To prevent this happening, you should prevent the dimming operation while the app is actively tracking a Wiimote. This is done using the PowerManager class. You first grab the global PowerManager using getSystemService(). Then you use it to create a "wake lock" by calling its newWakeLock method with appropriate arguments. This lock will be active until it is released and will prevent the display from blanking during that time. You release it by calling release() on the wake lock object.

You should start the wake lock after you find a Wii Remote, and keep it as long as tracking is happening, i.e. until the user clicks on "disconnect".

Making the device visible on a Mac

To connect your device to your MAC (OSX): Earlier versions of Mac OS should recognize the device by default, however newer versions might not do so. Here is a fix to that: Assuming you have the android SDK installed, you need to follow the steps mentioned here: connect to archos 5

This should make your device visible to you.


Grading

You will receive 20 points for a fully-functional app. That is, the app should connect to a Wii Remote, track several bright points in view, and show them on its display. The interface should remain live while its connecting. And the display should not blank while the app is running.

Extra credit: You can receive up to 10 points of extra credit for:

  • Having a mode that allows you to draw trails by remembering the previous point positions. An attractive rendering is "ghost trails" where each point decays over time back to black.
  • Drawing colors, ideally using the Wiimote to control the choice of color.
  • With two or more points, you can attempt to draw a stick figure. i.e. assume the two points are the arms, and extrapolate a head and legs. If there are three points you could model two arms and a neck or head, and extrapolate the rest. For light sources you could use flashlights, strap-on headlamps, or electronic "tea" lights.
  • Add Wii Motionplus support to Wiiuse/WiiuseJ. The basic data from the MotionPlus extension controller is roll-pitch-yaw rate from 3 gyroscopes.

Add a Link to Your Submission Here

Retrieved from "http://bid.berkeley.edu/cs160-fall10/index.php/Individual_Programming_Assignment_4"
Personal tools