-
Notifications
You must be signed in to change notification settings - Fork 89
Plowshare exposes an Application Programming Interface (API), so developers can easily create additional modules.
Here is the list of callable functions from modules. No other binary call (awk, grep, sed, ...) should be allowed. This is just a list. Some functions are detailed on Modules wiki page.
Note: All API calls are returning a comprehensive return value expect 'String manipulation' and 'HTML helper functions' family function. So, you can know if operation (or parsing) went wrong.
Proper error handling is really easy with plowshare, just don't forget to append '|| return' to your command! Notice that *_quiet functions are silent and does not handle errors (will always return 0), use them only when required.
HTTP requests:
-
curl: url ($1) -
curl_with_log: same as curl but force debug level verbose for this request. For upload function only. post_login
String manipulation:
-
replace,replace_all: stdin (input buffer), $1 (from string), $2 (to sting) -
strip: delete leading and trailing spaces -
lowercase,uppercase: string(s) ($1..$n) -
first_line,last_lineandnth_line: it's grep -
delete_first_line,delete_last_line: stdin (input buffer), n ($2, optional)
Text parsing with regexp:
-
matchandmatchi -
match_remote_url: string ($1), additional schemes ($2..$n, optional) -
parse,parse_quiet,parse_all,parse_all_quiet: filter regexp ($1), match regexp ($2), offset ($3, optional). Input buffer throughstdin. -
delete_filter_line: text data (stdin), stop regexp ($1), offset ($2, optional, default 0), startline ($3, optional, default 1)
File related:
-
create_tempfile: suffix ($1, optional) -
get_filesize: filename ($1) -
basename_file: filename ($1)
Logging:
-
log_debug: string ($1) -
log_error: string ($1)
HTTP parsing:
-
grep_http_header_location,grep_http_header_location_quiet grep_http_header_content_locationgrep_http_header_content_typegrep_http_header_content_lengthgrep_http_header_content_disposition-
parse_cookie,parse_cookie_quiet: entry name ($1)
HTML parsing:
-
parse_attr,parse_attr_quiet,parse_all_attr,parse_all_attr_quiet -
parse_tag,parse_tag_quiet,parse_all_tag,parse_all_tag_quiet -
grep_form_by_order: html data ($1), n ($2, optional) -
grep_script_by_order: html data ($1), n ($2, optional) grep_form_by_namegrep_form_by_idparse_form_action-
parse_form_input_by_id,parse_form_input_by_id_quiet -
parse_form_input_by_name,parse_form_input_by_name_quiet -
parse_form_input_by_type,parse_form_input_by_type_quiet
HTML helper functions:
-
break_html_lines,break_html_lines_alt: html data (stdin) -
strip_html_comments: html data (stdin) -
basename_url: input url ($1) html_to_utf8-
uri_encode,uri_encode_strict: second one is for relative URLs uri_decode
JSON parsing:
-
parse_json,parse_json_quiet match_json_true
Functions calling external tools
detect_javascriptjavascript-
captcha_process: image file ($1), captcha type/hint ($2) -
recaptcha_process: site public key ($1) -
solvemedia_captcha_process: site public key ($1) -
captcha_ack: id (given bycaptcha_processorrecaptcha_process) -
captcha_nack: id (given bycaptcha_processorrecaptcha_process)
Plowdown dedicated command line option function:
-
wait: related to--timeoutswitch
Plowlist dedicated command line option function:
-
list_submit: report list of url and list of filenames. link list ($1), name list ($2, optional) , link prefix string ($3, optional), link suffix string ($4, optional)
Plowprobe dedicated command line option function:
-
translate_size: convert a string ($1) with optional unit to an integer. For example:123.54MB.
Storage (module private file to save data):
-
storage_set: key ($1), strinvgalue ($2, optional: unset=delete key, set=replace value) -
storage_get: key ($1), return result on stdout -
storage_reset: delete all entries (and file) -
storage_timestamp_set: add__date__entry with current date -
storage_timestamp_diff: update-flag ($1, optional). Return value number in seconds (substracted with current date) on stdout.
Miscellaneous:
-
prompt_for_password: terminal string entry (not echoed) -
md5,md5_file: calculate MD5 hash (of $1 argument, string or file) -
sha1,sha1_file: calculate SHA-1 hash (of $1 argument, string or file) -
random: generate pseudo random sequences (numbers, strings) -
split_auth: string ($1). Splits ($1) with colon separator character. Use it to parse$AUTHor$AUTH_FREE.
Note: Plowshare API is completely backward compatible. To keep module sources simple, there is no minimum API version requirement variable or scheme.
- Add
delete_filter_lineAPI (see commit).
- Add second optional argument to
delete_last_lineAPI (see commit).
- Add
sha1_fileAPI (see commit).
- New error code
$ERR_EXPIRED_SESSION, use in conjunction with storage API (see commit). No additional API added.
-
list_submit: allow single name as second argument
- Add storage API.