The following wget command does a good job at recursively crawling the full domain, saving the downloaded files in a single folder, and then deleting it all:
wget --delete-after -r -nd http://www.example.com/
When run from the command line, this works perfectly. When run via PHP's exec (or system, shell_exec, passthru) as follows, it only fetches the index page, but seems to go no deeper than that:
exec('wget --delete-after -r -nd http://www.example.com/');
If this were a permissions issue, I'd think it wouldn't download the index page either, but it does (noticeable when I take out '--delete-after').
There's no robots.txt involved, and no output is shown if I pass it through echo. What am I missing?