0

I have a view:

def Processinitialscan(request):
    EnteredDomain = request.GET.get('domainNm')

    #get raw output
    getDomainLinksFromGoo = settings.GOOGLE_BASEURL_FOR_HARVEST+settings.GOO_RESULT_DOMAIN_QUERIED+EnteredDomain
    rawGatheredGooOutput = mechanizeBrowser.open(getDomainLinksFromGoo)

    beautifulSoupObj = BeautifulSoup(mechanizeBrowser.response().read()) #read the raw response
    getFirstPageLinks = beautifulSoupObj.find_all('cite') #get first page of urls

    pattern = re.compile('^.*start=')   #set regex to search on - find anything like: " <domain and path here>start= "
    getRemainingPageUrls = beautifulSoupObj.find_all('a',attrs={'class': 'fl', 'href': pattern})

    NumberOfUrlsFound = len(getRemainingPageUrls)

    MaxUrlsToGather = ((NumberOfUrlsFound*10)+settings.GOOGLE_RESULT_AMT_ACCOUNT_FOR_PAGE_1) # +10 because 10 represents the urls on the first page

    url_data = UrlData(NumberOfUrlsFound, pattern) 
    #return HttpResponse(MaxUrlsToGather)

    return render(request, 'VA/scan/process_scan.html', {
        'url_data':url_data,'EnteredDomain':EnteredDomain,'getDomainLinksFromGoo':getDomainLinksFromGoo,
        'getRemainingPageUrls' : getRemainingPageUrls, 'NumberOfUrlsFound':NumberOfUrlsFound,
        'getFirstPageLinks' : getFirstPageLinks, 'MaxUrlsToGather' : MaxUrlsToGather
    })

and a template:

{% block block_containercontent %}
    {% autoescape on %}
    <h1>{{ EnteredDomain }}</h1>
<strong>url used: </strong>{{ getDomainLinksFromGoo }}<br />
<hr>
<br>
<strong>first page of links</strong> {{ getFirstPageLinks }}
<hr>
<br><strong>number of "next" links</strong> {{ NumberOfUrlsFound }}
<hr>
<br>
<strong>remaining urls:</strong> {{ getRemainingPageUrls }}
    {% if url_data.num_of_urls > 1 %}
    {% for url in url_data.url_list %}
        {{ url }}
    {% endfor %}
{% endif %}
    {% endautoescape %}

{% endblock block_containercontent %}

This template outputs:

url used: https://www.google.com/search?q=site%3Aasite.com


first page of links [<cite>www.google.com/webmasters/</cite>, <cite>www.asite.com</cite>, <cite>www.asite.com/blog/</cite>, <cite>www.asite.com/blog/projects/</cite>, <cite>www.asite.com/blog/category/internet/</cite>, <cite>www.asite.com/blog/category/goals/</cite>, <cite>www.asite.com/blog/category/uncategorized/</cite>, <cite>www.asite.com/blog/why-i-left-facebook/2013/01/</cite>, <cite>www.asite.com/blog/category/startups-2/</cite>, <cite>www.asite.com/blog/category/goals/</cite>, <cite>www.asite.com/blog/category/internet/</cite>]


number of "next" links 2

My question: how do I utilize the NumberOfUrlsFound in a loop within the template to generate links like: /search?q=site:entereddomain.com&start=10, /search?q=site:entereddomain.com&start=20, then follow the links using beautifulsoup based on the value of NumberOfUrlsFound. So if NumberOfUrlsFound = 2, then urls search?q=site:asite.com&start=10, search?q=site:asite.com&start=20 should be generated, furthermore:

(puesdo code..):

if(NumberOfUrlsFound > 1)
    foreach(NumberOfUrlsFound)
        # generate url with start=n+10  
        ## asite.com?/search?start=10  
        ## Then ...
        ## asite.com?/search?start=20
        ## and so on ..
        # where n represents the previous number
        # this n number is determined by `NumberOfUrlsFound` which might have a value of 2 for example
        # this value of 2 represents a max value of start=20 value to generate urls on.
4

1 回答 1

1

You probably can create a data object to represent the data you want to display in the template.

class UrlData(object):
    def __init__(self, num_of_urls, url_pattern):
        self.num_of_urls = num_of_urls
        self.url_pattern = url_pattern

    def url_list(self):
        # Returns a list of strings that represent the urls you want based on num_of_urls
        # e.g. asite.com/?search?start=10
        urls = []
        for i in xrange(self.num_of_urls):
            urls.append(self.url_pattern + 'start=' + str((i + 1) * 10))
        return urls

In your views.py

# Create a UrlData object from NumberOfUrlsFound and a url_pattern
# url_pattern being the asite.com/?search?start=
url_data = UrlData(NumberOfUrlsFound, getDomainLinksFromGoogle) 

return render(request, template, {'url_data': url_data, ...})

Just create object with the data in your view function and pass the object to the template.

In your template you can do something like:

# Mirroring your check
{% if url_data.num_of_urls > 1 %} 
    # We'll iterate through the url_list created from the function defined in UrlData
    {% for url in url_data.url_list %}
         {{ url }} # asite.com/?search...
    {% endfor %}
{% endif %}

In the template, when you call url_data.url_list it'll run the function in UrlData

于 2013-10-16T22:13:24.570 回答