pyspider无法设置项目间隔重复执行

solidvacuum 2018-07-26

用pyspider设置的项目,爬取一静态网页的数据并存到mysql,在代码调试中点run成功执行,在控制面板中第一次执行也能成功,后面就不能自动运行了,不知道什么原因,现将项目源代码和控制板中的设置贴出来,哪位大神给指导一下,感激不尽!

#!/usr/bin/env python

# -*- encoding: utf-8 -*-

# Created on 2018-07-23 16:21:51

# Project: crawlnpm

 

from pyspider.libs.base_handler import *

import pymysql

 

class Handler(BaseHandler):

    crawl_config = {

    }

 

    @every(seconds=30)

    def on_start(self):

        self.crawl('http://bd.lcfuturecenter.com:8080/ReferRemainReportAction.do.html', callback=self.index_page,age=1)

 

    #config(age=10)

    def index_page(self, response):

        db=pymysql.connect(host='localhost',user='root',password='12345',port=3306,db='mysql')

        cursor=db.cursor()

        sql='insert into crawl(rush,triple,feederserial,exchangefrequency,currentlotname,emptytime,partsname,mcno,remain,mcname,linetable) values(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)'

        for part in response.doc('part').items():

            rush=part.children('rush').text()

            triple=part.children('triple').text()

            feederserial=part.children('feederserial').text()

            exchangefrequency=part.children('exchangefrequency').text()

            currentlotname=part.children('currentlotname').text()

            emptytime=part.children('emptytime').text()

            partsname=part.children('partsname').text()

            mcno=part.children('mcno').text()

            remain=part.children('remain').text()

            mcname=part.children('mcname').text()

            linetable=part.children('linetable').text()

            #print(currentlotname)

            try:

               cursor.execute(sql,(rush,triple,feederserial,exchangefrequency,currentlotname,emptytime,partsname,mcno,remain,mcname,linetable))

               db.commit()

            except:

               db.rollback()

               #for feederserial in part.children('feederserial').items():

                #print(feederserial.text())

            #for currentlotname in part.children('currentlotname').items():

             #print(part.children('currentlotname').text())

        

 

 

 

solidvacuum 2018-07-26
控制面板的设置:
[group] npm2 RUNNING 1/3 16.5+0.88
5m: 40
1h: 366
1d: 366
all: 3  
zhuifengsmile 2018-08-10
曳恳匚缸侵俳叶备叭党炯匚铣讯纲
zhuifengsmile 2018-08-10
侨湃张致讶赏促
mp3.baidu.com/songlist/545074356
辖参凶芯亩汲貌好
mp3.baidu.com/songlist/545092335
亚松史彩堪翁旁
mp3.baidu.com/songlist/545095032
芭估于压量崩挠
mp3.baidu.com/songlist/545632433
岳允暮翟翁渍孔肪焊烁彩澈松翁啄鹤废驮
http://mp3.baidu.com/songlist/545074356

http://mp3.baidu.com/songlist/545092335

http://mp3.baidu.com/songlist/545095032

http://mp3.baidu.com/songlist/545632433
拭炯炯拦以倨拷僮尚穆纶松,曳恳匚缸侵俳叶备叭党炯匚铣讯纲
zhuifengsmile 2018-08-10
曳恳匚缸侵俳叶备叭党炯匚铣讯纲
zhuifengsmile 2018-08-10

的口舌交锋已经被对方认定是能拿主意的人而他们想要罚金已经寻思着

http://mp3.baidu.com/songlist/545074356

 

http://mp3.baidu.com/songlist/545092335

 

http://mp3.baidu.com/songlist/545095032

 

http://mp3.baidu.com/songlist/545632433

Global site tag (gtag.js) - Google Analytics