Execute Computer Use Action
Executes a computer use action on the sandbox.
Path Parameters
The organization ID
All possible computer use actions, with optional callId
Click the mouse at the specified coordinates
"mouse:click"X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Mouse button flag combination. 1: left, 2: right, 4: middle, 8: back, 16: forward; add them together to press multiple buttons at once.
Key to hold down during click, in xdotool key syntax. Example: "ctrl", "alt", "alt+shift".
Double click the mouse at the specified coordinates
"mouse:doubleClick"X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Mouse button index
Key to hold down during double click, in xdotool key syntax. Example: "ctrl", "alt", "alt+shift".
Move the mouse to the specified coordinates
"mouse:move"X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Key to hold down during move, in xdotool key syntax. Example: "ctrl", "alt", "alt+shift".
Scroll the mouse
"mouse:scroll"X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Vertical scroll steps
Horizontal scroll steps
Key to hold down during scroll, in xdotool key syntax. Example: "ctrl", "alt", "alt+shift".
Drag the mouse from start to end coordinates
"mouse:drag"Start X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Start Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
End X coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
End Y coordinate
Length in pixels
Indicates the unit is pixel
"px"Length in pixels
Length as a fraction
"/"Numerator of the fraction
Denominator of the fraction
Key to hold down during drag, in xdotool key syntax. Example: "ctrl", "alt", "alt+shift".
Type text content
"keyboard:type"Text content to type
Whether to treat line breaks as enter. If true, any line breaks(\n) in content will be treated as enter key press, and content will be split into multiple lines.
falsePress a keyboard hotkey
"keyboard:hotkey"Hotkey combination, in xdotool key syntax. Examples: "a", "Return", "alt+Tab", "ctrl+s", "Up", "KP_0" (for the numpad 0 key).
Duration in milliseconds. If specified, the hotkey will be held for a while and then released.
1 <= value <= 5000Take a screenshot of the current screen
"screenshot"Wait for a specified duration
"wait"Duration in milliseconds
Indicates the action has finished
"finished"Completion message
Indicates the action has failed
"failed"Failure message
Indicates the human user should take over the control
"client:user-takeover"Press ONE key down, in xdotool key syntax. Only use this action if hotkey or type cannot satisfy your needs.
"key:down"Key to press
Release ONE key, in xdotool key syntax. Only use this action if keydown cannot satisfy your needs and only after a key down.
"key:up"Key to release
Whether to include the screenshot url after action in the response
trueWhether to include the cursor position after action in the response
trueResponse Body
{
"screenShot": "http://example.com",
"cursorPosition": {
"x": 0,
"y": 0,
"screenWidth": 0,
"screenHeight": 0,
"screenIndex": 0
}
}