2025-12-17 04:31:40 [scrapy.utils.log] INFO: Scrapy 2.11.1 started (bot: news_scraper) 2025-12-17 04:31:40 [scrapy.utils.log] INFO: Versions: lxml 6.0.2.0, libxml2 2.14.6, cssselect 1.3.0, parsel 1.10.0, w3lib 2.3.1, Twisted 25.5.0, Python 3.11.13 (main, Aug 12 2025, 22:39:41) [GCC 14.2.0], pyOpenSSL 25.3.0 (OpenSSL 3.5.3 16 Sep 2025), cryptography 46.0.1, Platform Linux-5.15.0-157-generic-x86_64-with 2025-12-17 04:31:40 [scrapy.addons] INFO: Enabled addons: [] 2025-12-17 04:31:40 [asyncio] DEBUG: Using selector: EpollSelector 2025-12-17 04:31:40 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor 2025-12-17 04:31:40 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop 2025-12-17 04:31:40 [scrapy.extensions.telnet] INFO: Telnet Password: dbc06ad8cd2d1ac2 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-call.apigateway to before-call.api-gateway 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/endpoints.json 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/sdk-default-configuration.json 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Event choose-service-name: calling handler 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/s3/2006-03-01/service-2.json.gz 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json.gz 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/partitions.json 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f94048ec9a0> 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler 2025-12-17 04:31:40 [botocore.endpoint] DEBUG: Setting s3 timeout as (60, 60) 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/_retry.json 2025-12-17 04:31:40 [botocore.client] DEBUG: Registering retry handlers for service: s3 2025-12-17 04:31:40 [botocore.utils] DEBUG: Registering S3 region redirector handler 2025-12-17 04:31:40 [botocore.utils] DEBUG: Registering S3Express Identity Resolver 2025-12-17 04:31:40 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.closespider.CloseSpider', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'scrapy.extensions.throttle.AutoThrottle'] 2025-12-17 04:31:40 [scrapy.crawler] INFO: Overridden settings: {'AUTOTHROTTLE_ENABLED': True, 'BOT_NAME': 'news_scraper', 'CLOSESPIDER_TIMEOUT': 1800, 'CONCURRENT_REQUESTS': 4, 'DOWNLOAD_DELAY': 2, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/opt/scrapyd/logs/news_scraper/thuvienphapluatdocs_timestamp/3192a4b4db0111f099e2d6783c969646.log', 'NEWSPIDER_MODULE': 'news_scraper.spiders', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['news_scraper.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'} 2025-12-17 04:31:40 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'news_scraper.middlewares.NewsScraperDownloaderMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-12-17 04:31:40 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-12-17 04:31:40 [scrapy.middleware] INFO: Enabled item pipelines: [] 2025-12-17 04:31:40 [scrapy.core.engine] INFO: Spider opened 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-call.apigateway to before-call.api-gateway 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search 2025-12-17 04:31:40 [botocore.hooks] DEBUG: Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section 2025-12-17 04:31:40 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/endpoints.json 2025-12-17 04:31:41 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/sdk-default-configuration.json 2025-12-17 04:31:41 [botocore.hooks] DEBUG: Event choose-service-name: calling handler 2025-12-17 04:31:41 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/s3/2006-03-01/service-2.json.gz 2025-12-17 04:31:41 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json.gz 2025-12-17 04:31:41 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/partitions.json 2025-12-17 04:31:41 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler 2025-12-17 04:31:41 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f94039618a0> 2025-12-17 04:31:41 [botocore.hooks] DEBUG: Event creating-client-class.s3: calling handler 2025-12-17 04:31:41 [botocore.endpoint] DEBUG: Setting s3 timeout as (60, 60) 2025-12-17 04:31:41 [botocore.loaders] DEBUG: Loading JSON file: /usr/local/lib/python3.11/site-packages/botocore/data/_retry.json 2025-12-17 04:31:41 [botocore.client] DEBUG: Registering retry handlers for service: s3 2025-12-17 04:31:41 [botocore.utils] DEBUG: Registering S3 region redirector handler 2025-12-17 04:31:41 [botocore.utils] DEBUG: Registering S3Express Identity Resolver 2025-12-17 04:31:41 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-12-17 04:31:41 [thuvienphapluatdocs_timestamp] INFO: Spider opened: thuvienphapluatdocs_timestamp 2025-12-17 04:31:41 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6031 2025-12-17 04:31:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): thuvienphapluat.vn:443 2025-12-17 04:31:41 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /robots.txt HTTP/1.1" 200 None 2025-12-17 04:31:41 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2025-12-17 04:31:41 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban-moi HTTP/1.1" 200 None 2025-12-17 04:31:41 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2025-12-17 04:31:41 [thuvienphapluatdocs_timestamp] INFO: ['https://thuvienphapluat.vn/van-ban/Cong-nghe-thong-tin/Quyet-dinh-3439-QD-BGDDT-2025-Khung-noi-dung-thi-diem-giao-duc-tri-tue-nhan-tao-cho-hoc-sinh-pho-thong-684660.aspx', 'https://thuvienphapluat.vn/van-ban/Doanh-nghiep/Nghi-dinh-320-2025-ND-CP-huong-dan-Luat-Thue-thu-nhap-doanh-nghiep-665051.aspx', 'https://thuvienphapluat.vn/van-ban/Tai-chinh-nha-nuoc/Thong-bao-691-TB-VPCP-2025-cong-tac-thanh-lap-Trung-tam-tai-chinh-quoc-te-tai-Viet-Nam-684563.aspx', 'https://thuvienphapluat.vn/van-ban/Xay-dung-Do-thi/Thong-bao-692-TB-VPCP-2025-ket-luan-cuoc-hop-khac-phuc-hau-qua-thien-tai-bao-lu-684562.aspx', 'https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-2715-QD-TTg-2025-thuc-hien-khuyen-nghi-doi-voi-Bao-cao-thuc-hien-Cong-uoc-quyen-dan-su-684726.aspx', 'https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Nghi-quyet-408-NQ-CP-2025-thuc-hien-cong-tac-bau-cu-dai-bieu-Quoc-hoi-khoa-XVI-684724.aspx', 'https://thuvienphapluat.vn/van-ban/Linh-vuc-khac/Quyet-dinh-5272-QD-BNNMT-2025-Ke-hoach-thi-diem-truy-xuat-nguon-goc-qua-sau-rieng-684662.aspx', 'https://thuvienphapluat.vn/van-ban/Lao-dong-Tien-luong/Quyet-dinh-2711-QD-TTg-2025-Chien-luoc-phat-trien-doi-ngu-tri-thuc-thoi-ky-day-manh-cong-nghiep-hoa-684725.aspx', 'https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-48-2025-QD-TTg-Quy-che-tiep-nhan-tren-He-thong-thong-tin-ve-van-ban-quy-pham-phap-luat-684723.aspx', 'https://thuvienphapluat.vn/van-ban/Bat-dong-san/Quyet-dinh-53-2025-QD-UBND-sua-doi-Quyet-dinh-35-2020-QD-UBND-Bang-gia-dat-Tay-Ninh-684697.aspx', 'https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4683-QD-BVHTTDL-2025-Ke-hoach-thuc-hien-Nghi-quyet-282-NQ-CP-684663.aspx', 'https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Nghi-dinh-319-2025-ND-CP-thu-tuc-trien-khai-co-che-phat-trien-khoa-hoc-cong-trinh-duong-sat-684661.aspx', 'https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-5252-QD-BNNMT-2025-cong-bo-thu-tuc-hanh-chinh-noi-bo-sua-doi-linh-vuc-thu-y-684653.aspx', 'https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Quyet-dinh-2287-QD-BXD-2025-Ke-hoach-thuc-hien-Quyet-dinh-1901-QD-TTg-684568.aspx', 'https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Thong-bao-687-TB-VPCP-2025-ket-luan-cuoc-hop-Ban-Chi-dao-ve-ket-qua-quan-ly-dieu-hanh-gia-10-thang-dau-684566.aspx', 'https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4676-QD-BVHTTDL-2025-bo-sung-danh-muc-cac-noi-dung-thi-dau-thuoc-nhom-III-684564.aspx', 'https://thuvienphapluat.vn/van-ban/Tai-nguyen-Moi-truong/Cong-dien-239-CD-TTg-2025-khan-truong-khac-phuc-hau-qua-thien-tai-phuc-hoi-san-xuat-kinh-doanh-684557.aspx', 'https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-48-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-tinh-Hung-Yen-684556.aspx', 'https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-47-2025-TT-BXD-vung-nuoc-cang-bien-khu-vuc-hang-hai-dia-phan-Gia-Lai-684555.aspx', 'https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-46-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-thanh-pho-Da-Nang-684554.aspx'] 2025-12-17 04:31:41 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Cong-nghe-thong-tin/Quyet-dinh-3439-QD-BGDDT-2025-Khung-noi-dung-thi-diem-giao-duc-tri-tue-nhan-tao-cho-hoc-sinh-pho-thong-684660.aspx HTTP/1.1" 200 None 2025-12-17 04:31:41 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:41 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Giao-thong-Van-tai/Thong-tu-46-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-thanh-pho-Da-Nang-684554.aspx HTTP/1.1" 200 None 2025-12-17 04:31:41 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Giao-thong-Van-tai/Thong-tu-47-2025-TT-BXD-vung-nuoc-cang-bien-khu-vuc-hang-hai-dia-phan-Gia-Lai-684555.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Giao-thong-Van-tai/Thong-tu-48-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-tinh-Hung-Yen-684556.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Tai-nguyen-Moi-truong/Cong-dien-239-CD-TTg-2025-khan-truong-khac-phuc-hau-qua-thien-tai-phuc-hoi-san-xuat-kinh-doanh-684557.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/The-thao-Y-te/Quyet-dinh-4676-QD-BVHTTDL-2025-bo-sung-danh-muc-cac-noi-dung-thi-dau-thuoc-nhom-III-684564.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bo-may-hanh-chinh/Thong-bao-687-TB-VPCP-2025-ket-luan-cuoc-hop-Ban-Chi-dao-ve-ket-qua-quan-ly-dieu-hanh-gia-10-thang-dau-684566.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Giao-thong-Van-tai/Quyet-dinh-2287-QD-BXD-2025-Ke-hoach-thuc-hien-Quyet-dinh-1901-QD-TTg-684568.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bo-may-hanh-chinh/Quyet-dinh-5252-QD-BNNMT-2025-cong-bo-thu-tuc-hanh-chinh-noi-bo-sua-doi-linh-vuc-thu-y-684653.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Giao-thong-Van-tai/Nghi-dinh-319-2025-ND-CP-thu-tuc-trien-khai-co-che-phat-trien-khoa-hoc-cong-trinh-duong-sat-684661.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/The-thao-Y-te/Quyet-dinh-4683-QD-BVHTTDL-2025-Ke-hoach-thuc-hien-Nghi-quyet-282-NQ-CP-684663.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bat-dong-san/Quyet-dinh-53-2025-QD-UBND-sua-doi-Quyet-dinh-35-2020-QD-UBND-Bang-gia-dat-Tay-Ninh-684697.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bo-may-hanh-chinh/Quyet-dinh-48-2025-QD-TTg-Quy-che-tiep-nhan-tren-He-thong-thong-tin-ve-van-ban-quy-pham-phap-luat-684723.aspx HTTP/1.1" 200 None 2025-12-17 04:31:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:42 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 07:40:00, url: https://thuvienphapluat.vn/van-ban/Cong-nghe-thong-tin/Quyet-dinh-3439-QD-BGDDT-2025-Khung-noi-dung-thi-diem-giao-duc-tri-tue-nhan-tao-cho-hoc-sinh-pho-thong-684660.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 07:40:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 15:23:00, url: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-46-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-thanh-pho-Da-Nang-684554.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-46-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-thanh-pho-Da-Nang-684554.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 15:08:00, url: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-47-2025-TT-BXD-vung-nuoc-cang-bien-khu-vuc-hang-hai-dia-phan-Gia-Lai-684555.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-47-2025-TT-BXD-vung-nuoc-cang-bien-khu-vuc-hang-hai-dia-phan-Gia-Lai-684555.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 14:17:00, url: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-48-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-tinh-Hung-Yen-684556.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Thong-tu-48-2025-TT-BXD-cong-bo-vung-nuoc-cang-bien-thuoc-dia-phan-tinh-Hung-Yen-684556.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 09:38:00, url: https://thuvienphapluat.vn/van-ban/Tai-nguyen-Moi-truong/Cong-dien-239-CD-TTg-2025-khan-truong-khac-phuc-hau-qua-thien-tai-phuc-hoi-san-xuat-kinh-doanh-684557.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Tai-nguyen-Moi-truong/Cong-dien-239-CD-TTg-2025-khan-truong-khac-phuc-hau-qua-thien-tai-phuc-hoi-san-xuat-kinh-doanh-684557.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 16:50:00, url: https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4676-QD-BVHTTDL-2025-bo-sung-danh-muc-cac-noi-dung-thi-dau-thuoc-nhom-III-684564.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4676-QD-BVHTTDL-2025-bo-sung-danh-muc-cac-noi-dung-thi-dau-thuoc-nhom-III-684564.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 09:39:00, url: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Thong-bao-687-TB-VPCP-2025-ket-luan-cuoc-hop-Ban-Chi-dao-ve-ket-qua-quan-ly-dieu-hanh-gia-10-thang-dau-684566.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Thong-bao-687-TB-VPCP-2025-ket-luan-cuoc-hop-Ban-Chi-dao-ve-ket-qua-quan-ly-dieu-hanh-gia-10-thang-dau-684566.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 13:28:00, url: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Quyet-dinh-2287-QD-BXD-2025-Ke-hoach-thuc-hien-Quyet-dinh-1901-QD-TTg-684568.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Quyet-dinh-2287-QD-BXD-2025-Ke-hoach-thuc-hien-Quyet-dinh-1901-QD-TTg-684568.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 13:28:00, url: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-5252-QD-BNNMT-2025-cong-bo-thu-tuc-hanh-chinh-noi-bo-sua-doi-linh-vuc-thu-y-684653.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-5252-QD-BNNMT-2025-cong-bo-thu-tuc-hanh-chinh-noi-bo-sua-doi-linh-vuc-thu-y-684653.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 07:37:00, url: https://thuvienphapluat.vn/van-ban/Giao-thong-Van-tai/Nghi-dinh-319-2025-ND-CP-thu-tuc-trien-khai-co-che-phat-trien-khoa-hoc-cong-trinh-duong-sat-684661.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 07:37:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 15:39:00, url: https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4683-QD-BVHTTDL-2025-Ke-hoach-thuc-hien-Nghi-quyet-282-NQ-CP-684663.aspx 2025-12-17 04:31:43 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/The-thao-Y-te/Quyet-dinh-4683-QD-BVHTTDL-2025-Ke-hoach-thuc-hien-Nghi-quyet-282-NQ-CP-684663.aspx 2025-12-17 04:31:43 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Lao-dong-Tien-luong/Quyet-dinh-2711-QD-TTg-2025-Chien-luoc-phat-trien-doi-ngu-tri-thuc-thoi-ky-day-manh-cong-nghiep-hoa-684725.aspx HTTP/1.1" 200 None 2025-12-17 04:31:43 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:43 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Linh-vuc-khac/Quyet-dinh-5272-QD-BNNMT-2025-Ke-hoach-thi-diem-truy-xuat-nguon-goc-qua-sau-rieng-684662.aspx HTTP/1.1" 200 None 2025-12-17 04:31:43 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:43 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bo-may-hanh-chinh/Nghi-quyet-408-NQ-CP-2025-thuc-hien-cong-tac-bau-cu-dai-bieu-Quoc-hoi-khoa-XVI-684724.aspx HTTP/1.1" 200 None 2025-12-17 04:31:43 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Bo-may-hanh-chinh/Quyet-dinh-2715-QD-TTg-2025-thuc-hien-khuyen-nghi-doi-voi-Bao-cao-thuc-hien-Cong-uoc-quyen-dan-su-684726.aspx HTTP/1.1" 200 None 2025-12-17 04:31:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Xay-dung-Do-thi/Thong-bao-692-TB-VPCP-2025-ket-luan-cuoc-hop-khac-phuc-hau-qua-thien-tai-bao-lu-684562.aspx HTTP/1.1" 200 None 2025-12-17 04:31:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Tai-chinh-nha-nuoc/Thong-bao-691-TB-VPCP-2025-cong-tac-thanh-lap-Trung-tam-tai-chinh-quoc-te-tai-Viet-Nam-684563.aspx HTTP/1.1" 200 None 2025-12-17 04:31:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: https://thuvienphapluat.vn:443 "GET /van-ban/Doanh-nghiep/Nghi-dinh-320-2025-ND-CP-huong-dan-Luat-Thue-thu-nhap-doanh-nghiep-665051.aspx HTTP/1.1" 200 None 2025-12-17 04:31:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://thuvienphapluat.vn/van-ban-moi) 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 07:39:00, url: https://thuvienphapluat.vn/van-ban/Bat-dong-san/Quyet-dinh-53-2025-QD-UBND-sua-doi-Quyet-dinh-35-2020-QD-UBND-Bang-gia-dat-Tay-Ninh-684697.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 07:39:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 07:59:00, url: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-48-2025-QD-TTg-Quy-che-tiep-nhan-tren-He-thong-thong-tin-ve-van-ban-quy-pham-phap-luat-684723.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 07:59:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 08:12:00, url: https://thuvienphapluat.vn/van-ban/Lao-dong-Tien-luong/Quyet-dinh-2711-QD-TTg-2025-Chien-luoc-phat-trien-doi-ngu-tri-thuc-thoi-ky-day-manh-cong-nghiep-hoa-684725.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 08:12:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 15:38:00, url: https://thuvienphapluat.vn/van-ban/Linh-vuc-khac/Quyet-dinh-5272-QD-BNNMT-2025-Ke-hoach-thi-diem-truy-xuat-nguon-goc-qua-sau-rieng-684662.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Linh-vuc-khac/Quyet-dinh-5272-QD-BNNMT-2025-Ke-hoach-thi-diem-truy-xuat-nguon-goc-qua-sau-rieng-684662.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 09:36:00, url: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Nghi-quyet-408-NQ-CP-2025-thuc-hien-cong-tac-bau-cu-dai-bieu-Quoc-hoi-khoa-XVI-684724.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 09:36:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 16:53:00, url: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-2715-QD-TTg-2025-thuc-hien-khuyen-nghi-doi-voi-Bao-cao-thuc-hien-Cong-uoc-quyen-dan-su-684726.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Bo-may-hanh-chinh/Quyet-dinh-2715-QD-TTg-2025-thuc-hien-khuyen-nghi-doi-voi-Bao-cao-thuc-hien-Cong-uoc-quyen-dan-su-684726.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 13:06:00, url: https://thuvienphapluat.vn/van-ban/Xay-dung-Do-thi/Thong-bao-692-TB-VPCP-2025-ket-luan-cuoc-hop-khac-phuc-hau-qua-thien-tai-bao-lu-684562.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Xay-dung-Do-thi/Thong-bao-692-TB-VPCP-2025-ket-luan-cuoc-hop-khac-phuc-hau-qua-thien-tai-bao-lu-684562.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-16 09:38:00, url: https://thuvienphapluat.vn/van-ban/Tai-chinh-nha-nuoc/Thong-bao-691-TB-VPCP-2025-cong-tac-thanh-lap-Trung-tam-tai-chinh-quoc-te-tai-Viet-Nam-684563.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-16 is out of date range: from 2025-12-17 to 2025-12-17, skipping article: https://thuvienphapluat.vn/van-ban/Tai-chinh-nha-nuoc/Thong-bao-691-TB-VPCP-2025-cong-tac-thanh-lap-Trung-tam-tai-chinh-quoc-te-tai-Viet-Nam-684563.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: Min timestamp: 2025-12-17 10:15:00, published timestamp: 2025-12-17 10:10:00, url: https://thuvienphapluat.vn/van-ban/Doanh-nghiep/Nghi-dinh-320-2025-ND-CP-huong-dan-Luat-Thue-thu-nhap-doanh-nghiep-665051.aspx 2025-12-17 04:31:44 [thuvienphapluatdocs_timestamp] INFO: 2025-12-17 10:10:00 smaller than 2025-12-17 10:15:00 2025-12-17 04:31:44 [scrapy.core.engine] INFO: Closing spider (finished) 2025-12-17 04:31:44 [boto3.s3.transfer] DEBUG: Opting out of CRT Transfer Manager. Preferred client: auto, CRT available: False, Instance Optimized: False. 2025-12-17 04:31:44 [boto3.s3.transfer] DEBUG: Using default client. pid: 189239, thread: 140273601669944 2025-12-17 04:31:44 [s3transfer.utils] DEBUG: Acquiring 0 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: UploadSubmissionTask(transfer_id=0, {'transfer_future': }) about to wait for the following futures [] 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: UploadSubmissionTask(transfer_id=0, {'transfer_future': }) done waiting for dependent futures 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: Executing task UploadSubmissionTask(transfer_id=0, {'transfer_future': }) with kwargs {'client': , 'config': , 'osutil': , 'request_executor': , 'transfer_future': } 2025-12-17 04:31:44 [s3transfer.futures] DEBUG: Submitting task PutObjectTask(transfer_id=0, {'bucket': 'dagster-output-data', 'key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'extra_args': {}}) to executor for transfer request: 0. 2025-12-17 04:31:44 [s3transfer.utils] DEBUG: Acquiring 0 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: PutObjectTask(transfer_id=0, {'bucket': 'dagster-output-data', 'key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'extra_args': {}}) about to wait for the following futures [] 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: PutObjectTask(transfer_id=0, {'bucket': 'dagster-output-data', 'key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'extra_args': {}}) done waiting for dependent futures 2025-12-17 04:31:44 [s3transfer.tasks] DEBUG: Executing task PutObjectTask(transfer_id=0, {'bucket': 'dagster-output-data', 'key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'extra_args': {}}) with kwargs {'client': , 'fileobj': , 'bucket': 'dagster-output-data', 'key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'extra_args': {}} 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-endpoint-resolution.s3: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-endpoint-resolution.s3: calling handler > 2025-12-17 04:31:44 [botocore.regions] DEBUG: Calling endpoint provider with parameters: {'Bucket': 'dagster-output-data', 'Region': 'us-east-1', 'UseFIPS': False, 'UseDualStack': False, 'Endpoint': 'https://lake-api.actable.ai/', 'ForcePathStyle': True, 'Accelerate': False, 'UseGlobalEndpoint': True, 'Key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True} 2025-12-17 04:31:44 [botocore.regions] DEBUG: Endpoint provider result: https://lake-api.actable.ai/dagster-output-data 2025-12-17 04:31:44 [s3transfer.utils] DEBUG: Releasing acquire 0/None 2025-12-17 04:31:44 [botocore.regions] DEBUG: Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2025-12-17 04:31:44 [botocore.regions] DEBUG: Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 's3', 'disableDoubleEncoding': True} 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.handlers] DEBUG: Adding expect 100 continue header to request. 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.34.57 md/Botocore#1.34.162 ua/2.0 os/linux#5.15.0-157-generic md/arch#x86_64 lang/python#3.11.13 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.162', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Expect': '100-continue'}, 'body': , 'auth_path': '/dagster-output-data/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'url': 'https://lake-api.actable.ai/dagster-output-data/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': True, 'auth_type': 'v4', 's3_redirect': {'redirected': False, 'bucket': 'dagster-output-data', 'params': {'Bucket': 'dagster-output-data', 'Key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl', 'Body': }}, 'input_params': {'Bucket': 'dagster-output-data', 'Key': 'thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl'}, 'signing': {'region': 'us-east-1', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'us-east-1'}]}}} 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler > 2025-12-17 04:31:44 [botocore.auth] DEBUG: Calculating signature using v4 auth. 2025-12-17 04:31:44 [botocore.auth] DEBUG: CanonicalRequest: PUT /dagster-output-data/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl content-md5:1B2M2Y8AsgTpgAmY7PhCfg== host:lake-api.actable.ai x-amz-content-sha256:UNSIGNED-PAYLOAD x-amz-date:20251217T043144Z content-md5;host;x-amz-content-sha256;x-amz-date UNSIGNED-PAYLOAD 2025-12-17 04:31:44 [botocore.auth] DEBUG: StringToSign: AWS4-HMAC-SHA256 20251217T043144Z 20251217/us-east-1/s3/aws4_request 0e87eb2863d586c9f740decfeea838b2205b247fcf0943c544585963dc63843f 2025-12-17 04:31:44 [botocore.auth] DEBUG: Signature: 8bdc01dc292d1fa659081a822da3d27a428e61b5aad1dae0fab351ad253fb848 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.endpoint] DEBUG: Sending http request: 2025-12-17 04:31:44 [botocore.httpsession] DEBUG: Certificate path: /usr/local/lib/python3.11/site-packages/certifi/cacert.pem 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): lake-api.actable.ai:443 2025-12-17 04:31:44 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response. 2025-12-17 04:31:44 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body. 2025-12-17 04:31:44 [urllib3.connectionpool] DEBUG: https://lake-api.actable.ai:443 "PUT /dagster-output-data/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl HTTP/1.1" 200 0 2025-12-17 04:31:44 [botocore.parsers] DEBUG: Response headers: {'Server': 'nginx/1.24.0 (Ubuntu)', 'Date': 'Wed, 17 Dec 2025 04:31:44 GMT', 'Content-Length': '0', 'Connection': 'keep-alive', 'Accept-Ranges': 'bytes', 'ETag': '"d41d8cd98f00b204e9800998ecf8427e"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin, Accept-Encoding', 'X-Amz-Bucket-Region': 'us-east-1', 'X-Amz-Id-2': 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8', 'X-Amz-Request-Id': '1881E671B07A06AC', 'X-Content-Type-Options': 'nosniff', 'X-Ratelimit-Limit': '25637', 'X-Ratelimit-Remaining': '25637', 'X-Xss-Protection': '1; mode=block'} 2025-12-17 04:31:44 [botocore.parsers] DEBUG: Response body: b'' 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler 2025-12-17 04:31:44 [botocore.retryhandler] DEBUG: No retry needed. 2025-12-17 04:31:44 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler > 2025-12-17 04:31:44 [s3transfer.utils] DEBUG: Releasing acquire 0/None 2025-12-17 04:31:44 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (0 items) in: s3://dagster-output-data/thuvienphapluatdocs_timestamp/thuvienphapluatdocs_timestamp_3192a4b4db0111f099e2d6783c969646_scheduled_2025-12-17.jl 2025-12-17 04:31:44 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/response_bytes': 7938613, 'downloader/response_count': 22, 'downloader/response_status_count/200': 22, 'elapsed_time_seconds': 3.867649, 'feedexport/success_count/S3FeedStorage': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 12, 17, 4, 31, 44, 815014, tzinfo=datetime.timezone.utc), 'log_count/DEBUG': 155, 'log_count/INFO': 53, 'memusage/max': 124141568, 'memusage/startup': 124141568, 'request_depth_max': 1, 'response_received_count': 22, 'robotstxt/request_count': 1, 'robotstxt/response_count': 1, 'robotstxt/response_status_count/200': 1, 'scheduler/dequeued': 21, 'scheduler/dequeued/memory': 21, 'scheduler/enqueued': 21, 'scheduler/enqueued/memory': 21, 'start_time': datetime.datetime(2025, 12, 17, 4, 31, 40, 947365, tzinfo=datetime.timezone.utc)} 2025-12-17 04:31:44 [scrapy.core.engine] INFO: Spider closed (finished)