为什么这是一个未处理的承诺拒绝？

Question

我试图重写我的代码以正确使用承诺。

完整的程序应该是从t恤网站刮取数据。第一块代码应该进入网站的首页，抓取可立即使用的产品页面，然后将URL存储在一个数组中。其余的URL将被存储在“剩余”中，以便稍后执行第二次刮取。

目前手动的单元测试每个部分：

//TASK: Create a command line application that goes to an ecommerce site to get the latest prices. 
    //Save the scraped data in a spreadsheet (CSV format). 



//Modules being used: 
var cheerio = require('cheerio'); 
var request = require('request'); 

//harcoded url 
var url = 'http://shirts4mike.com/'; 

//url for tshirt pages 
var urlSet = new Set(); 

var remainder; 

    const requestPromise = function(url) { 
    return new Promise(function(resolve, reject) { 
     request(url, function(error, response, html) { 

      if(error) return reject(error); 

      if(!error && response.statusCode == 200){ 
       return resolve(html); 
      }  
     }); 
    }); 
} 


function firstScrape (url) { 
    return requestPromise(url) 
     .then(function(html) { 
      var $ = cheerio.load(html); 

      var links = []; 

      //get all the links 
      $('a[href*=shirt]').each(function(){ 
       var a = $(this).attr('href'); 

       //add into link array 
       links.push(url + a); 

      }); 
      return links; 
      // return this array you've made 
     }); 
} 


function nextStep (arrayOfLinks) { 
    var promiseArray = []; 

    for(var link in arrayOfLinks){ 
     promiseArray.push(requestPromise(link)); 
     return Promise.all(promiseArray); 
    }     
} 


function lastStep (arrayOfHTMLresults){ 
    for(var html in arrayOfHTMLresults){ 
     var $ = cheerio.load(html); 

     //if page has a submit it must be a product page 
     if($('[type=submit]').length !== 0){ 

      //add page to set 
      urlSet.add(scrapeLink); 

     } else if(remainder == undefined) { 
      //if not a product page, add it to remainder so it another scrape can be performed. 
      remainder = scrapeLink;       
     } 
    } 
    console.log(urlSet); 
    console.log(remainder); 
} 


firstScrape(url) 
    .then(nextStep) 
    .then(lastStep); 

目前，我发现了以下错误：

(node:71094) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: Invalid URI "0"

这是我想promisify代码：

// Load front page of shirts4mike 
function firstScrape(){ 
    request(url, function(error, response, html) { 
     if(!error && response.statusCode == 200){ 
      var $ = cheerio.load(html); 

     //iterate over links with 'shirt' 
      $('a[href*=shirt]').each(function(){ 
       var a = $(this).attr('href'); 

       //create new link 
       var scrapeLink = url + a; 

       //for each new link, go in and find out if there is a submit button. 
       //If there, add it to the set 
       request(scrapeLink, function(error,response, html){ 
        if(!error && response.statusCode == 200) { 
         var $ = cheerio.load(html); 

         //if page has a submit it must be a product page 
         if($('[type=submit]').length !== 0){ 

          //add page to set 
          urlSet.add(scrapeLink); 

         } else if(remainder == undefined) { 
          //if not a product page, add it to remainder so it another scrape can be performed. 
          remainder = scrapeLink;       
         } 
        } 
       }); 
      });  
     } 
    }); 
} 

我不能解决的是我该如何使用urlSet.add(scrapeLink);在lastStep()的时候它不知道什么scrapeLink是？

任何想法为什么？谢谢

Answer 1

.add()不是Array.prototype方法，你也returnpromiseArray内for循环，而不是推Promise到promiseArray和使用Promise.all()

function nextStep (arrayOfLinks) { 
    var promiseArray = []; 

    for(var i = 0; i < arrayOfLinks.length; i++) { 
     var link = requestPromise(arrayOfLinks[i]); 
     promiseArray.push(link); 
    } 

    return Promise.all(promiseArray)     
} 

Answer 2

UPDATE由于质疑变化：

所以从firstScrape()你可以返回一个结果对象，而不仅仅是一个链接数组：

return { scrapeLink: link, result: links } 

那么您需要获得在nextStep()作为承诺的结果，在那里你可以再次与相同形状的返回的东西：在lastStep()代替arrayOfHTMLresults

return { scrapeLink: firstStepResult.scrapLink, result: Promise.all(promiseArray) } 

然后获得通过你将不得不一个对象，它看起来像：

{ scrapeLink: "http://someurl.com", result: arrayOfHTMLresults } 

以前的答案：

您将需要在for ... in循环中初始化您的变量。例如与const,var或let取决于你的用例和JS版本。

for(var link in arrayOfLinks){ 
    promiseArray.add(requestPromise(link)); 
    return promiseArray; 
}