Extracted URLs

#Overview

This plugin identifies and lists all unique HTTP and HTTPS URLs discovered within the unzipped contents of an IPA file.

#How It Works

The plugin starts by targeting the unzipped directory, where the IPA's contents are extracted. It then leverages the powerful command-line tool, rg (ripgrep), to search for URLs. The specific command executed is:

rg --text --no-filename -oP '(https?://\S+)' "<unzippedPath>" > "<tempExtractedUrlsFilePath>"
  • --text: Ensures that even binary files are searched for text patterns.

  • --no-filename: Prevents the filename from being prefixed to each match, keeping the output clean.

  • -oP '(https?://\\S+)': This is the core of the extraction. It uses a Perl-compatible regular expression to output only the matched URL patterns. The regex (https?://\\S+) specifically looks for strings starting with http:// or https:// followed by any sequence of non-whitespace characters.

The output from this rg command is redirected and saved to a temporary file, temp-extracted-urls.txt.

After the initial search, the plugin reads this temporary file. It then goes through a crucial cleanup process for each extracted URL. This involves systematically removing common trailing characters like <, ", ', `, spaces, and backslashes, which might appear due to surrounding text or the broad nature of the regex match.

All the cleaned URLs are then passed through a Set to automatically eliminate any duplicate entries, ensuring that only unique URLs are retained. Finally, these unique, verified URLs are written to the permanent file, extracted-urls.txt, within your scan directory, with each URL on a new line. The temporary file is then deleted to maintain cleanliness.

When you request the data, the plugin simply reads the contents ofextracted-urls.txt, splits it by newlines, and provides an array of the extracted URLs. If the file is absent or empty, it gracefully returns an empty array.

#Example Output

[
  "https://api.example.com/data",
  "http://cdn.anotherdomain.net/assets/image.png",
  "https://www.thirdparty.com/auth",
  "https://updates.myapp.com"
]