HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonModerate

IP and router connections

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
codereviewweb-scrapingstackoverflowpythonregex
androuterconnections

Problem

How can I make my code more pythonic ? I definitely think there is a way to make this code a lot more readable and clear + shorter...

But I haven't found an effective way. Any techniques I can use to make it more concise and clear?

```
#!/usr/bin/env python
import requests
import socket
import random
import re
import time

link = "https://100.1.X.X/testpage.html"
ipAddresses = ['100.1.177.1', '100.1.177.2', '100.1.177.3']
superprice = 31867
medprice = 22992
gutdop = 12000

targetsuper = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]
targetmedium = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]
targetknife = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]
targetdoppler = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]
targetm = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]
targetmg = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]

def SaleProcess(cart, cookie_value):
print "=========FOUND SOMETHING========="
print cart
print cookie_value
def main():
x = 0
while True:
try:
proc_c = cookie_value
start = Scanner(cookie_value)
except:
print "= Sleeping for 1sec and trying again"
time.sleep(1)
continue

def Scanner(cookie_value):
x = 0
while x (.*?)<"
analystid = re.findall(regexp, strResponce)
regexp = 'addToCart\((.*?)\)'
price = re.findall(regexp, strResponce)
regexp = 'image\/(.*?)\/256fx256f'
image = re.findall(regexp, strResponce)

price1,analystid1,image1 = price[:1],analystid[:1],image[:1]
price2,analystid2,image2 = price[1:2],analystid[1:2],image[1:2]
price3,analystid3,image3 = price[2:3],analystid[2:3],image[2:3]
price4,analystid4,image4 = price[3:4],analystid[3:4],image[3:4]
price5,an

Solution

Unused variables

x, proc_c, start and JustDOit are set but never used, thus they are unneeded. For instance, write:

def main():
    while True:
        try:
            Scanner(cookie_value)
        except:
            print "= Sleeping for 1sec and trying again"
            time.sleep(1)


by the way, where does cookie_value come from? It is used all around your code but never set anywhere.
Flawed control flow

In your main function, what exceptions are you expecting exactly? Not even talking about the bare except that will handles exceptions that you are expecting as well as those you are not, but Scanner already performs exception management so why expect anything else to happen?

Also, you have a while True: in main and something that is pretty close in Scanner. This seems all too confusing. I’d rather have Scanner perform only one iteration over the IP addresses and let the while loop in the main.
Too much variables

The amount of variables in Scanner has already been discussed by @MaximGalushka. It’s approach to reduce the amount of copy pasting is pretty good, you can put all your second try .. except .. logic into his Parsed object constructor for maximum efficiency.

But you also happen to have all your targetXXX being the same without modifying them at any point. Why not use only one, then?
Flawed logic

You’re iterating over IP addresses but not using them to get any content. Instead, you’re always getting content from link. You probably want to define

LINK = "https://{}/testpage.html"


and then get the content using

requests.get(LINK.format(ipAddress))


You are also checking if cartX == cartidX way too many times. Check it once and if it fails there is no need to perform other checks.
Variable naming

Use proper case to differentiate between purpose of variables. TitleCase is usually for classes: use lowercase or lower_snake_case for function names. Use UPPERCASE for constants.
Proposed improvement

Using @MaximGalushka concept:

#!/usr/bin/env python
import requests
import socket
import random
import re
import time

LINK = 'https://{}/testpage.html'
IP_ADDRESSES = ['100.1.177.1', '100.1.177.2', '100.1.177.3']
SUPER_PRICE = 31867
MED_PRICE = 22992
GUTDOP = 12000

TARGETS = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]

class Parsed:
    def __init__(self, price, analist_id, image):
        cart, analyst = str(analyst_id).split(',')
        self.cart = cart[3:-1]
        self.analyst = analyst[2:-3]
        self.price = str(price).strip('[]').replace('\'', '')
        self.cart_id, total_price = self.price.split(',')
        self.total_price = float(total_price)
        self.image = str(image)[2:-3]

    def analyse(self):
        if self.cart == self.cart_id:
            self.analyse_price(TARGETS, 3000) # targetknife
            self.analyse_image(TARGETS, TARGETS, TARGETS, SUPER_PRICE, GUTDOP) # targetdoppler, targetmg, targetm
            self.analyse_price(TARGETS, SUPER_PRICE) # targetsuper
            self.analyse_price(TARGETS, MED_PRICE) # targetmedium

    def analyse_price(self, target, limit_price):
        if any(string in self.analyst for string in target):
            if self.total_price (.*?)<", response_str)
        price = re.findall('addToCart\((.*?)\)', response_str)    
        image = re.findall('image\/(.*?)\/256fx256f', response_str)

        try:
            parsed_objects = [Parsed(price[i], analyst[i], image[i]) for i in xrange(10)]
        except ValueError:
            print "Cannot parse content from", address
            continue

        for object in parsed_objects:
            object.analyse()

        time.sleep(5)

def main():
    while True:
        parse_content()

if __name__ == "__main__":
    main()

Code Snippets

def main():
    while True:
        try:
            Scanner(cookie_value)
        except:
            print "= Sleeping for 1sec and trying again"
            time.sleep(1)
LINK = "https://{}/testpage.html"
requests.get(LINK.format(ipAddress))
#!/usr/bin/env python
import requests
import socket
import random
import re
import time


LINK = 'https://{}/testpage.html'
IP_ADDRESSES = ['100.1.177.1', '100.1.177.2', '100.1.177.3']
SUPER_PRICE = 31867
MED_PRICE = 22992
GUTDOP = 12000

TARGETS = ["QHTxDZ7I56KU", "bXH5ApeO4", "tLu8JAllx8zJfA", "LwJ2UUGcVs_ns"]

class Parsed:
    def __init__(self, price, analist_id, image):
        cart, analyst = str(analyst_id).split(',')
        self.cart = cart[3:-1]
        self.analyst = analyst[2:-3]
        self.price = str(price).strip('[]').replace('\'', '')
        self.cart_id, total_price = self.price.split(',')
        self.total_price = float(total_price)
        self.image = str(image)[2:-3]

    def analyse(self):
        if self.cart == self.cart_id:
            self.analyse_price(TARGETS, 3000) # targetknife
            self.analyse_image(TARGETS, TARGETS, TARGETS, SUPER_PRICE, GUTDOP) # targetdoppler, targetmg, targetm
            self.analyse_price(TARGETS, SUPER_PRICE) # targetsuper
            self.analyse_price(TARGETS, MED_PRICE) # targetmedium

    def analyse_price(self, target, limit_price):
        if any(string in self.analyst for string in target):
            if self.total_price <= limit_price:
                self.process()

    def analyse_image(self, global_target, target_img_1, target_img_2, limit_price, gutdop):
        if any(word in self.analyst for word in global_target):
            if self.image in target_img_1:
                if self.total_price <= limit_price:
                    self.process()
            if self.image in target_img_2:
                if self.total_price <= gutdop:
                    self.process()

    def process(self):
        print "=========FOUND SOMETHING========="
        print self.cart


def parse_content():
    for ip in IP_ADDRESSES:
        address = LINK.format(ip)
        try:
            response = request.get(address)
        except requests.exceptions.RequestException:
            print "Can not fetch content from", address
            continue
        else:
            response_str = response.text

        analyst = re.findall("&item=(.*?)'>(.*?)<", response_str)
        price = re.findall('addToCart\((.*?)\)', response_str)    
        image = re.findall('image\/(.*?)\/256fx256f', response_str)

        try:
            parsed_objects = [Parsed(price[i], analyst[i], image[i]) for i in xrange(10)]
        except ValueError:
            print "Cannot parse content from", address
            continue

        for object in parsed_objects:
            object.analyse()

        time.sleep(5)

def main():
    while True:
        parse_content()

if __name__ == "__main__":
    main()

Context

StackExchange Code Review Q#108691, answer score: 11

Revisions (0)

No revisions yet.