qaml uses AI to turn your natural language into app automations. qaml actions are used for device level control. However, there are additional controls you can specify in the prompt.

Please use this guide as best practices when issuing commands to qaml.

Controls

Long Press

Long press the <button name> button

This is the command to tap and hold on a region. By default, this holds down for 3 seconds.

Example:

q.execute("Long press the record button")

Tapping Images

Images are not identified at this time

Images are treated as a black box in vision mode. The model knows that there is an image and it knows the coordinates, but it does not know what the image is.

Aside from the wait_until and assert commands, qaml cannot see images within an app. Note, this is not true if you are using accessibility mode. Accessibility mode uses device level labels which may include image descriptions.

We recommend you use placement queues to interact with specific images. Note, icon buttons are not images and can be interacted with directly.

Example:

# assert uses a vision model to see and evaluate the screen
q.asssert("There is a carousel of images of coffee tables") 

# execute does not use vision on images. Use labels and positioning to interact with images.
q.execute("Tap the image under the header Coffee Tables")