A pinball machine that is able to play itself, it’s as simple as that. Well, I say it’s simple, it’s actually extremely complicated. This was a four-person team project with some of my fellow graduating friends at KSU, and all of it was finished in one semester as our senior design project.
LIMITATIONS & CHALLENGES
There was LOTS of stuff to learn on this project, and it’s definitely the most ambitious I’ve had so far. Thankfully, I had a team with me on this one. 3 Big challenges, (1) Modeling and building the physical machine, (2) Wiring a very complicated system in an easy-to-diagnose sort of way, (3) Writing the software to manage the low-level stuff and to make autonomy decisions.
Overview:
As mentioned above, this project was a collaboration between me and 3 other guys for our senior design project. It was completed over the course of a semester, and turned out exceptionally well, we were all super proud of it. A quick overview of the project is below.
The pinball machine is composed of eighteen LEDs, fourteen switches (two are stand-up targets and one is a spinner), seven solenoids (two are flippers, three are pop bumpers, and two are slingshots). The system has four main components that interact with each other in simple ways.
Computer. The computer executes the real-time processing for the system and controls when the LEDs should be turned on, flippers should be flipped, scores to add, etc. For every ball interaction with a component, the computer should know about it and make decisions based on that information. In addition to the playfield switches, the computer also receives information from a camera mounted above the playfield. This camera will let the computer know where the ball is at all times to be used in flipping timing.
Microcontroller. The microcontroller acts as a pass-through between the playfield components and the computer. Any switch that is triggered or flipped on the field is registered to the I/O and passed as a ROS message to the computer. The microcontroller also takes ROS messages from the computer, turns on LEDs, and actuates the flippers.
Custom Circuit Boards. The circuit boards handle the interaction between the microcontroller and the physical components. These boards are responsible for taking in a control signal from the microcontroller and outputting a power signal to individual components. This happens via transistors acting as a switch.
Playfield. The playfield consists of a CNC’d piece of 0.750” thick baltic birch plywood with holes and slots end milled from the bottom for component mounting. These include lights, switches, targets, bumpers, flippers, slingshots, etc. These components are the mediums through which the user can interact with the machine and where information, in the form of flashing LEDs, is output.
The Mechanical System:
The mechanical system is incredibly intricate and fully rendered in SolidWorks. It was CNC’d on a 5-axis CNC machine at Kennesaw State University. This gave us some really nice accuracy in the dimensions and the final product looked beautiful. All the acrylic that is on the machine was lazer cut at Georgia Tech and was also modeled in SolidWorks.
One of the main design features we wanted was the ability to open up the machine easily and look under the hood. This was so we could diagnose problems that we knew were going to come up later when we hooked together all the electrical parts. The final design sports a rotational piece near the player that pivots the entire system upright and allows for easy access to the electrical system.
The Electrical System:
The electrical system is fairly simple at it’s core, but gets scaled up to quite a large number of inputs and outputs. Here’s the basic concept:
The circuit for the switches
The circuit for the lights
The circuit for the solenoid coils
The circuit for the flipper coils
For each of these circuits, a 5V control signal is sent from the arduino to actually control each transistor. This control signal is originally generated from the autonomy system that runs on the laptop connected through a serial connection to the arudino.
The Software System:
The software system is where I spend most of my time and efforts. The whole system is built on ROS and is a mixture of Python and C++. Below is a general overview of what messages pass between what nodes and how everything is connected.
Four main nodes communicate with each other asynchronously that control the flow of the pinball machine when it is run in autonomous mode. These nodes are Input_Output.ino, track_metal.cpp, run_low_level.py, and GUI.py. When not running in autonomous mode, the node track_metal.cpp may be omitted.
Input_Output.ino is the script uploaded to the Arduino that handles all the literal switching on/off of the pinball components and reading in from any playfield switches. These items include flippers, bumpers, slingshots, lights, switches, stand-up targets, a spinner, a start button, and an autonomy on/off switch. The Input_Output.ino node has two major functions: turning on items on the playfield and relaying the status of switches back to the computer.
track_metal.cpp node takes care of locating the ball in real-time, locating the flippers at set increments, and making a decision of when to flip the flippers. Additionally, the track_metal.cpp node takes in a single input from the Input_Output.ino node of the status of the physical autonomy switch to know when it should be flipping the flippers.
Locating the ball in track_metal.cpp is done in 6 steps:
1.Take a picture of the playfield when the flippers are down and there is no ball in the frame.
2.Convert that frame to grayscale.
3.Compare every new frame to the original frame by using a binary threshold. This is possible because each frame has a single pixel color value that defines it.
4.Enlarge anomalies found in the binary comparison using cv::dilate().
5.Retrieve a list of contours from this binary comparison using cv::findContours().
6.Loop through the list. Any contour’s area that is in-between some threshold will be classified as a ball.
This method for finding pinballs on the playfield has both advantages and disadvantages over attempting to detect circles in the image. The first major advantage with this method is speed. After preliminary testing, trying to find circles in the image proved to have too much delay in the system and the pinballs would move too fast to be detected at high speeds. Secondly, when a pinball is moving at high speeds, it appears on the camera with a large amount of blur surrounding it. This could greatly throw off the detection of circles as the shape no longer appears circular. The major disadvantage of this was the frequent mislabeling of other objects as pinballs. Many times when lights would turn on or other things would move on the playfield they would be categorized as a ball. This, however, proved not to be a major issue as they would not typically appear in the flipping area. If we were to include velocity tracking over time of a specific pinball, this would have been a greater issue.
Locating the flippers in track_metal.cpp is done in 6 steps:
1.Create a HSV threshold that the flippers must fall between. Because they are bright yellow, it can be a relatively narrow hue threshold.
2.Enlarge anomalies found in the binary comparison using cv::dilate().
3.Retrieve a list of contours from this binary comparison using cv::findContours().
4.Find the centroid of each contour in the list.
5.Sort the centroids by their distance to the center of the frame.
6.Return the two most central contours as long as their areas are between a certain threshold. They will be the two flippers.
This method for finding the flippers was extremely effective and accurate. The only issue we had when implementing this on the real system was the introduction of the yellow acrylic. This acrylic fell into almost the same HSV color space that the flippers did. The above-mentioned steps, however, take the center two contours, which the flippers always were.
Lastly, the track_metal.cpp node tells the system to flip the flippers when the detected pinball falls into a desired “flip zone” that is attached to the flippers. The “flip zone” is created by initializing a circle on the center of rotation of each flipper, turning it into a N-gon with a custom function, then removing most of the points of that N-gon to create a wedge. This method allowed the user to tune the center of the circle location, the radius of the circle, and which points to remove from the circle. This tuning allowed certain shots on the playfield to be hit more frequently or less frequently and favoring different types of playstyles.
run_low_level.py node is responsible for, in software, keeping track of the status of all playfield components. This includes physical components like the switches and lights, but also digital components like the score, mode, and bonus multipliers. Every switch on the playfield has a ROS callback that triggers when the switch is hit by a pinball. These switch callbacks all mostly do the same thing such as updating the score and turning on a light. Some of the callbacks, like the switch that activates multiball, are slightly more complicated.
In addition to keeping track of all the playfield components in near-real-time, the run_low_level.py node is in charge of telling the Arduino to turn lights and solenoids on and off. To do this effectively, we make use of a python scheduling package called apscheduler. This package allows the developer to provide a ‘datetime’ formatted time to schedule a command or function to run. It does not, however, allow for a developer to say they want a command to be run ‘x seconds in the future’. We wrapped this functionality into run_low_level.py to allow for lights to be turned on and told to turn off in ‘x seconds’.
run_low_level.py also provides some functionality to keep track of a list of high scores for manual play. After each game, users can enter their name and it will check their score against a list internally. If they make it into the top ten players, their name is slotted into the list. To store and retrieve this list, the python pickle package was used for writing a serialized list to a file. Additionally, the list is initialized to a set of default scores if the user has not created the list before.
GUI.py is extremely simple compared to the others listed. The only thing that GUI.py does is display on the screen the current score, bonus, and play mode. All three of these enter GUI.py by a ROS topic that updates the GUI object. The GUI was built using the python Tkinter package. This package is incredibly simple to use and provides a sleek look.
Note: The content and the pictures in this article are contributed by the author. The opinions expressed by contributors are their own and not those of PCBWay. If there is any infringement of content or pictures, please contact our editor (zoey@pcbway.com) for deleting.
Written by
