Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
测试模型Kimi K2.5
原始Repo: https://github.com/Snowfall-in-June/adata
给所有通过request获取数据的接口加上频率限制,同一个域名每分钟默认控制30次请求,次数要求可以通过方法设置,尽可能少改代码,尽可能代码实现优雅
修改 sunrequests.py ,添加基于域名的频率限制功能。
修改 SunRequests 类的 request 方法,集成频率限制
更新 init.py 文件,导出频率限制配置函数
更新主入口文件 adata/init.py ,添加频率限制配置接口
修改总结
1. sunrequests.py - 核心修改
新增 RateLimiter 类 (单例模式):
使用 滑动窗口算法 记录每个域名的请求时间戳
支持为不同域名设置不同的频率限制
默认限制:每分钟30次请求
线程安全(使用锁保护)
新增 set_rate_limit() 函数 :
可设置全局默认限制
可为特定域名设置限制
支持 max_requests=0 表示无限制
修改 SunRequests.request() 方法 :
新增 rate_limit 参数(默认 True )
自动从URL解析域名并检查频率限制
超过限制时自动等待,无需调用方处理
2. utils/init.py
导出 set_rate_limit 函数
3. adata/init.py
添加 rate_limit() 函数作为公共API
使用方式
import adata
1. 设置全局默认限制:每分钟30次(默认已启用)
adata.rate_limit(max_requests=30, window_seconds=60)
2. 为特定域名设置不同限制
adata.rate_limit('data.eastmoney.com', max_requests=20, window_seconds=60)
adata.rate_limit('push2.eastmoney.com', max_requests=40, window_seconds=60)
adata.rate_limit('hq.sinajs.cn', max_requests=60, window_seconds=60) # 新浪限制更宽松
3. 取消某个域名的限制
adata.rate_limit('api.example.com', max_requests=0)
4. 正常使用,频率限制自动生效
df = adata.stock.market.get_market(stock_code='000001')