8. API - System Peripheral Devices¶

The pi-top Python SDK provides classes which represent devices, including some that can be used by generic devices, such as USB cameras. These classes are intended to simplify using these common system peripheral devices.

8.1. USB Camera¶

This class provides an easy way to:

save image and video files
directly access camera frames
process frames in the background (via callback)

It is easy to make use of some pre-written video processors, such as motion detection.

It is also possible to make use of this class to read frames from a directory of images, removing the need for a stream of images from physical hardware. This can be useful for testing, or simulating a real camera.

from time import sleep

from pitop import Camera

# Record a 10s video to ~/Camera/

cam = Camera()

cam.start_video_capture()
sleep(10)
cam.stop_video_capture()

By default, camera frames are of PIL.Image.Image type (using the Pillow module), which provides a standardized way of working with the image. These Image objects use raw, RGB-ordered pixels.

It is also possible to use OpenCV standard format, if desired. This may be useful if you are intending to do your own image processing with OpenCV. The OpenCV format uses raw, BGR-ordered pixels in a NumPy numpy.ndarray object. This can be done by setting the camera’s format property to “OpenCV”:

from pitop import Camera

c = Camera()
c.format = "OpenCV"

This can be also be done by passing the format to the camera’s constructor:

from pitop import Camera

c = Camera(format="OpenCV")

8.1.1. Using a USB Camera to Access Image Data¶

from pitop import Camera

cam = Camera()

while True:
    image = cam.get_frame()
    print(image.getpixel((0, 0)))

8.1.2. Using a USB Camera to Capture Video¶

from time import sleep

from pitop import Camera

# Record a 10s video to ~/Camera/

cam = Camera()

cam.start_video_capture()
sleep(10)
cam.stop_video_capture()

8.1.3. Adding Motion Detection to a USB Camera¶

from datetime import datetime
from time import localtime, sleep, strftime

from pitop import Camera

# Example code for Camera
# Records videos of any motion captured by the camera

cam = Camera()

last_motion_detected = None


def motion_detected():
    global last_motion_detected

    last_motion_detected = datetime.now().timestamp()

    if cam.is_recording() is False:
        print("Motion detected! Starting recording...")
        output_file_name = f"/home/pi/Desktop/My Motion Recording {strftime('%Y-%m-%d %H:%M:%S', localtime(last_motion_detected))}.avi"
        cam.start_video_capture(output_file_name=output_file_name)

        while (datetime.now().timestamp() - last_motion_detected) < 3:
            sleep(1)

        cam.stop_video_capture()
        print(f"Recording completed - saved to {output_file_name}")


print("Motion detector starting...")
cam.start_detecting_motion(
    callback_on_motion=motion_detected, moving_object_minimum_area=350
)

sleep(60)

cam.stop_detecting_motion()
print("Motion detector stopped")

8.1.4. Processing Camera Frame¶

from PIL import ImageDraw

from pitop import Camera

cam = Camera()


def draw_red_cross_over_image(im):
    # Use Pillow to draw a red cross over the image
    draw = ImageDraw.Draw(im)
    draw.line((0, 0) + im.size, fill=128, width=5)
    draw.line((0, im.size[1], im.size[0], 0), fill=128, width=5)
    return im


im = draw_red_cross_over_image(cam.get_frame())
im.show()

8.1.5. Processing Camera Frame Stream with OpenCV (Convert to grayscale)¶

from time import sleep

import cv2

from pitop import Camera

cam = Camera(format="OpenCV")


def show_gray_image(image):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    cv2.imshow("frame", gray)
    cv2.waitKey(1)  # Necessary to show image


# Use callback function for 60s
cam.on_frame = show_gray_image
sleep(60)


# Use get_frame indefinitely
try:
    while True:
        show_gray_image(cam.get_frame())

except KeyboardInterrupt:
    cv2.destroyAllWindows()

8.1.6. Ball Color Detection with OpenCV¶

from signal import pause

import cv2

from pitop.camera import Camera
from pitop.processing.algorithms import BallDetector


def process_frame(frame):
    detected_balls = ball_detector(frame, color=["red", "green", "blue"])

    red_ball = detected_balls.red
    if red_ball.found:
        print(f"Red ball center: {red_ball.center}")
        print(f"Red ball radius: {red_ball.radius}")
        print(f"Red ball angle: {red_ball.angle}")
        print()

    green_ball = detected_balls.green
    if green_ball.found:
        print(f"Green ball center: {green_ball.center}")
        print(f"Green ball radius: {green_ball.radius}")
        print(f"Green ball angle: {green_ball.angle}")
        print()

    blue_ball = detected_balls.blue
    if blue_ball.found:
        print(f"Blue ball center: {blue_ball.center}")
        print(f"Blue ball radius: {blue_ball.radius}")
        print(f"Blue ball angle: {blue_ball.angle}")
        print()

    cv2.imshow("Image", detected_balls.robot_view)
    cv2.waitKey(1)


ball_detector = BallDetector()
camera = Camera(resolution=(640, 480))
camera.on_frame = process_frame

pause()

8.1.7. Class Reference: USB Camera¶

class pitop.camera.Camera(index=None, resolution=(640, 480), camera_type=<CameraTypes.USB_CAMERA: 0>, path_to_images='', format='PIL', flip_top_bottom: bool = False, flip_left_right: bool = False, rotate_angle=0, name='camera')[source]¶

Provides a variety of high-level functionality for using the PMA USB Camera, including capturing images and video, and processing image data from the camera.

Parameters:	index (int) – ID of the video capturing device to open. Passing None will cause the backend to autodetect the available video capture devices and attempt to use them.

capture_image(output_file_name='')[source]¶

Capture a single frame image to file.

Note

If no output_file_name argument is provided, images will be stored in ~/Camera.

Parameters:	output_file_name (str) – The filename into which to write the image.

current_frame(format=None)[source]¶

Returns the latest frame captured by the camera. This method is non- blocking and can return the same frame multiple times.

By default the returned image is formatted as a PIL.Image.Image.

Parameters:	format (string) – DEPRECATED. Set ‘camera.format’ directly, and call this function directly instead.

format¶

classmethod from_file_system(path_to_images: str)[source]¶: Alternative classmethod to create an instance of a Camera object using a FileSystemCamera

classmethod from_usb(index=None)[source]¶: Alternative classmethod to create an instance of a Camera object using a UsbCamera

get_frame(format=None)[source]¶

Returns the next frame captured by the camera. This method blocks until a new frame is available.

Parameters:	format (string) – DEPRECATED. Set ‘camera.format’ directly, and call this function directly instead.

is_detecting_motion()[source]¶: Returns True if motion detection mode is enabled.

is_recording()[source]¶: Returns True if recording mode is enabled.

own_state¶: Representation of an object state that will be used to determine the current state of an object.

start_detecting_motion(callback_on_motion, moving_object_minimum_area=300)[source]¶

Begin processing image data from the camera, attempting to detect motion. When motion is detected, call the function passed in.

Warning

The callback function can take either no arguments or only one, which will be used to provide the image back to the user when motion is detected. If a callback with another signature is received, the method will raise an exception.

Parameters:	callback_on_motion (function) – A callback function that will be called when motion is detected. moving_object_minimum_area (int) – The sensitivity of the motion detection, measured as the area of pixels changing between frames that constitutes motion.

start_handling_frames(callback_on_frame, frame_interval=1, format=None)[source]¶

Begin calling the passed callback with each new frame, allowing for custom processing.

Warning

The callback function can take either no arguments or only one, which will be used to provide the image back to the user. If a callback with another signature is received, the method will raise an exception.

Parameters:	callback_on_frame (function) – A callback function that will be called every `frame_interval` camera frames. frame_interval (int) – The callback will run every frame_interval frames, decreasing the frame rate of processing. Defaults to 1. format (string) – DEPRECATED. Set ‘camera.format’ directly, and call this function directly instead.

start_video_capture(output_file_name='', fps=20.0, resolution=None)[source]¶

Begin capturing video from the camera.

Note

If no output_file_name argument is provided, video will be stored in ~/Camera.

Parameters:	output_file_name (str) – The filename into which to write the video. fps (int or float) – The framerate to use for the captured video. Defaults to 20.0 fps resolution (tuple) – The resolution to use for the captured video. Defaults to (640, 368)

stop_detecting_motion()[source]¶

Stop running the motion detection processing.

Does nothing unless start_detecting_motion has been called.

stop_handling_frames()[source]¶

Stops handling camera frames.

Does nothing unless start_handling_frames has been called.

stop_video_capture()[source]¶

Stop capturing video from the camera.

Does nothing unless start_video_capture has been called.

8.2. Keyboard Button¶

This class makes it easy to handle a keyboard button in the same way as a GPIO-based button.

You can listen for any standard keyboard key input. For example, using a or A will provide the ability to ‘listen’ for the A-key being pressed - with or without shift.

Warning

This class depends on pynput, which interfaces with Xorg to handle key press events. This means that this component cannot be used via SSH, or in a headless environment (that is, without a desktop environment).

Note

The DISPLAY environment variable is required to be set in order for this component to work.

Note

If your code is being run from a terminal window, then the key presses will be captured in the terminal output. This can cause confusion and issues around reading output.

from time import sleep

from pitop import KeyboardButton


def on_up_pressed():
    print("up pressed")


def on_up_released():
    print("up released")


def on_down_pressed():
    print("down pressed")


def on_down_released():
    print("down released")


def on_left_pressed():
    print("left pressed")


def on_left_released():
    print("left released")


def on_right_pressed():
    print("right pressed")


def on_right_released():
    print("right released")


keyboard_btn_up = KeyboardButton("up")
keyboard_btn_down = KeyboardButton("down")
keyboard_btn_left = KeyboardButton("left")
keyboard_btn_right = KeyboardButton("right")
keyboard_btn_uppercase_z = KeyboardButton("Z")

# Methods will be called when key is pressed:

keyboard_btn_up.when_pressed = on_up_pressed
keyboard_btn_up.when_released = on_up_released
keyboard_btn_down.when_pressed = on_down_pressed
keyboard_btn_down.when_released = on_down_released
keyboard_btn_left.when_pressed = on_left_pressed
keyboard_btn_left.when_released = on_left_released
keyboard_btn_right.when_pressed = on_right_pressed
keyboard_btn_right.when_released = on_right_released

# Or alternatively you can "poll" for key presses:

while True:
    if keyboard_btn_uppercase_z.is_pressed is True:
        print("Z pressed!")

    sleep(0.1)

8.2.1. Class Reference: KeyboardButton¶

class pitop.keyboard.KeyboardButton(key)[source]¶

is_pressed¶

Get or set the button state as a boolean value.

Return type:	bool

when_pressed¶

Get or set the ‘when pressed’ button state callback function. When set, this callback function will be invoked when this event happens.

Parameters:	callback (Function) – Callback function to run when a button is pressed.

when_released¶

Get or set the ‘when released’ button state callback function. When set, this callback function will be invoked when this event happens.

Parameters:	callback (Function) – Callback function to run when a button is released.

8.2.2. Special Key Names¶

You can listen for the following special keys by passing their names when creating an instance of KeyboardButton.

Identifier	Description
`alt`	A generic Alt key. This is a modifier.
`alt_l`	The left Alt key. This is a modifier.
`alt_r`	The right Alt key. This is a modifier.
`alt_gr`	The AltGr key. This is a modifier.
`backspace`	The Backspace key.
`caps_lock`	The CapsLock key.
`cmd`	A generic command button.
`cmd_l`	The left command button. On PC keyboards, this corresponds to the Super key or Windows key, and on Mac keyboards it corresponds to the Command key. This may be a modifier.
`cmd_r`	The right command button. On PC keyboards, this corresponds to the Super key or Windows key, and on Mac keyboards it corresponds to the Command key. This may be a modifier.
`ctrl`	A generic Ctrl key. This is a modifier.
`ctrl_l`	The left Ctrl key. This is a modifier.
`ctrl_r`	The right Ctrl key. This is a modifier.
`delete`	The Delete key.
`down`	A down arrow key.
`up`	An up arrow key.
`left`	A left arrow key.
`right`	A right arrow key.
`end`	The End key.
`enter`	The Enter or Return key.
`esc`	The Esc key.
`home`	The Home key.
`page_down`	The PageDown key.
`page_up`	The PageUp key.
`shift`	A generic Shift key. This is a modifier.
`shift_l`	The left Shift key. This is a modifier.
`shift_r`	The right Shift key. This is a modifier.
`space`	The Space key.
`tab`	The Tab key.
`insert`	The Insert key. This may be undefined for some platforms.
`menu`	The Menu key. This may be undefined for some platforms.
`num_lock`	The NumLock key. This may be undefined for some platforms.
`pause`	The Pause/Break key. This may be undefined for some platforms.
`print_screen`	The PrintScreen key. This may be undefined for some platforms.
`scroll_lock`	The ScrollLock key. This may be undefined for some platforms.
`f1`	The F1 key
`f2`	The F2 key
`f3`	The F3 key
`f4`	The F4 key
`f5`	The F5 key
`f6`	The F6 key
`f7`	The F7 key
`f8`	The F8 key
`f9`	The F9 key
`f10`	The F10 key
`f11`	The F11 key
`f12`	The F12 key
`f13`	The F13 key
`f14`	The F14 key
`f15`	The F15 key
`f16`	The F16 key
`f17`	The F17 key
`f18`	The F18 key
`f19`	The F19 key
`f20`	The F20 key