INNER CODE UNIT · Python

fetch_rss_feeds

zillionare/zillionare · scripts/enhanced_news_crawler.py:119

    def fetch_rss_feeds(self) -> List[NewsArticle]:
        """获取RSS源中的文章列表"""
        articles = []
        sources = self.config.get("sources", [])

        for source in sources:
            try:
                logger.info(f"Fetching from {source['name']}")
                articles.extend(self._fetch_single_source(source))

                delay = self.config.get("crawler_config", {}).get(
                    "delay_between_requests", 1
                )
                time.sleep(delay)

            except Exception as e:
                logger.error(f"Error fetching from {source['name']}: {e}")
                continue

View source record →