esieabot-ai.rst

esieabot-ai
===========

.. image:: esieabot-ai.jpg
    :width: 400
    :align: center

esieabot-ai is an optional library that makes it easy to use image recognition functions on your esieabot. Currently, it can detect and locate ArUco markers. It is still under development.

.. warning:: Currently, the distance measurement (Z axis) is calibrated, but not the other axes. Please check your measurements before using them.

ArUco marker
------------

.. image:: ArUco.png
    :width: 200
    :align: center

ArUco markers are simplistic markers used in augmented reality applications. Each marker represents a number, in this case esieabot-ai recognizes 4 by 4 ArUco markers, covering numbers from 0 to 250. Thanks to image processing, the position and orientation angle of the markers is recognized. This makes it possible to geolocate the esieabot in space using only its camera. You can generate markers on this site: https://chev.me/arucogen/.

.. warning:: For image recognition to work properly, the markers must be exactly 53mm square, with a small white band around them.

Installation
------------

To install esieabot-ai, simply run the following command: ``sudo apt install esieabot-ai-api``.

.. warning:: esieabot-ai-api cannot run in parallel with another application using the camera, such as esieabot-webcamera. You must therefore uninstall it with sudo apt remove esieabot-webcamera before continuing.

Test
----

You can test image recognition by logging on to the following address from your web browser: http://<address_of_your_esieabot>/ai/get_image

.. note:: Your esieabot's address is its IP address, which you can obtain with the command ``ip a``. If you're connected to its hotspot, the address is always 10.42.0.1. Otherwise, you'll need to look up the ip address of your robot on your network.

To find out which ArUco markers are detected, open the following page: http://<your_robot_address>/ai/get_markers

.. warning:: It is not possible to use the get_image function and the get_markers function at the same time. So you can't display what the camera sees and at the same time request the list of markers.

.. note:: It takes about 300ms to process a complete image. You need to take this delay into account in your programs.

Debugging
---------

If the tests don't work, you can go to the ``/esieabot/logs`` folder to find a ``esieabot-ai-api.py.log`` file. This file contains error messages which may be of interest to you. If it doesn't exist, esieabot-ai hasn't been installed correctly. In any case, if the application doesn't work, it's probably because you have another application monopolizing the camera.

Use in C
--------

You can use the esieabot-ai library directly in a C program like this :

.. code-block:: C
    
    #include <stdio.h>
    #include <stdlib.h>
    #include <esieabot-ai.h>

    int main() {
        while (1) {
            marker *markers = get_markers();
            while (markers->id != -1) {
                printf("Marker found: number %d at %dcm distance. x=%d y=%d\n", markers->id, markers->z, markers->x, markers->y);
                markers++;
            }
        }
        return 0;
    }

This simple program loops esieabot-ai to retrieve the list of markers as an array and iterate through it. To compile it, add ``-lesieabot-ai`` to the compile command.


Use in Python
-------------

You can also use the esieabot-ai library in python. However, you will have to make a request yourself.
To do this, you must first import the `requests` library.


.. code-block:: python
    
    import requests
    import math

    response = requests.get('http://127.0.0.1:5001/get_markers').json()

    for marker in response["markers"]:
        distance = math.sqrt(marker["x"] ** 2 + marker["y"] ** 2 + marker["z"] ** 2)
        print("Marker found: number {} at {}cm distance. x={} y={} z={}".format(marker["id"], distance, marker["x"], marker["y"], marker["z"]))


This program will, as in the C example, retrieve the list of markers, but in the form of a dictionary. The program will thus iterate in it for each element.


Structures
``````````

- ``marker`` is a structure containing the following fields:
    - ``int id`` is the marker id
    - ``int x`` is its position on the X axis in cm
    - ``int y`` is its position on the Y axis in cm
    - ``int z`` is its position on the Z axis in cm
    - ``int pitch`` is its rotation (pitch axis) in degrees
    - ``int roll`` is its rotation (roll axis) in degrees
    - ``int yaw`` is its rotation (yaw axis) in degrees

Angles are returned in degrees, between -180 and 180.

.. note:: A marker facing up and aligned perpendicular to the camera will have a roll of 180° but a pitch and yaw of 0°


Orientation
-----------

.. image:: yaw-pitch-roll.png
    :width: 400
    :align: center