[php]通过yield方法,批量multi_curl请求接口数据

yield是php5.5以后的新特性,详细可通过之前的博客了解一般的yield概念,这里采用异步的方式,整合所有的curl请求,收集再一起,
最后通过统一的curl_multi_*方法批量获取,代码如下

<?php
class Test {
    
    protected $_mh;
    protected $_ch;
    protected $_doMulti = true;
    
    
    public function mytest() {
       
        $gens = array();
        //test接口 请添加自己的接口
        $url   = array();
        $url[] = "https://www.baidu.com/";
        $url[] = "http://www.haosou.com/";
        $i = 3;
        while($i) {
            $gens[$i] = $this->_asyncQuery($url[1]);
            $i --;
        }
        echo "i am doing same other!";
        $gens[4] = $this->_asyncQuery($url[0]);   
        do {
            $running = false;
            foreach($gens as $id => $gen) {
                if ($gen->valid()) {
                    $running = true;
                    $gen->next();
                    $tmp = $gen->current();
                    if ($tmp !== null) {
                        $ret[$id] = $tmp;
                    }
                }
            }
        }while($running);
        var_dump($ret);
    }
    
    
    protected function _asyncQuery($url, $body = array()) {
    
        yield null; // create a generator
        //{{{产生ch
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);       
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // 302 redirect
        //产生ch结束
    
        $body = http_build_query($body);
    
        curl_setopt($ch, CURLOPT_POST, (bool) $body);
    
        if($body) {
            curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
        }
    
        //一次multi
        if(empty($this->_mh)) {
            $this->_mh = curl_multi_init();
        }
        curl_multi_add_handle($this->_mh, $ch);
        $this->_ch[] = $ch;
        
        yield null;
        //这里需要统一处理,后面的迭代器不需要再次curl_mutlti_exec  
        if($this->_doMulti) {
        do {
            $mrc = curl_multi_exec($this->_mh, $active);
            } while ($active > 0);
            //var_dump("this just do once");
        }
         
        $ch  = array_shift($this->_ch);
        $raw = curl_multi_getcontent($ch);
        curl_multi_remove_handle($this->_mh, $ch);
        //如果为空的话,关掉multi_close
        if(empty($this->_ch)){
            curl_multi_close($this->_mh);
        }
        $this->_doMulti = false;
    
        yield $raw;
    
    } 
}

可以通过microtime(true)计算一下执行时间,和统一的curl_multi方法一致。大约是单个curl循环调用时间的N分之一。N为请求数(前提是每个请求的时间都差不多,否则multi_curl受最长时间影响)

发表评论

电子邮件地址不会被公开。 必填项已用*标注


*