Gradio-Lite: Serverless Gradio Running Entirely in Your Browser
•
1
Abubakar Abid PRO
abidlabs
AI & ML interests
self-supervised learning, applications to medicine & biology, interpretation, reproducibility
Articles
Organizations
Posts
12
Post
1925
Open Models vs. Closed APIs for Software Engineers
-----------------------------------------------------------------------
If you're an ML researcher / scientist, you probably don't need much convincing to use open models instead of closed APIs -- open models give you reproducibility and let you deeply investigate the model's behavior.
But what if you are a software engineer building products on top of LLMs? I'd argue that open models are a much better option even if you are using them as APIs. For at least 3 reasons:
1) The most obvious reason is reliability of your product. Relying on a closed API means that your product has a single point-of-failure. On the other hand, there are at least 7 different API providers that offer Llama3 70B already. As well as libraries that abstract on top of these API providers so that you can make a single request that goes to different API providers depending on availability / latency.
2) Another benefit is eventual consistency going local. If your product takes off, it will be more economical and lower latency to have a dedicated inference endpoint running on your VPC than to call external APIs. If you've started with an open-source model, you can always deploy the same model locally. You don't need to modify prompts or change any surrounding logic to get consistent behavior. Minimize your technical debt from the beginning.
3) Finally, open models give you much more flexibility. Even if you keep using APIs, you might want to tradeoff latency vs. cost, or use APIs that support batches of inputs, etc. Because different API providers have different infrastructure, you can use the API provider that makes the most sense for your product -- or you can even use multiple API providers for different users (free vs. paid) or different parts of your product (priority features vs. nice-to-haves)
-----------------------------------------------------------------------
If you're an ML researcher / scientist, you probably don't need much convincing to use open models instead of closed APIs -- open models give you reproducibility and let you deeply investigate the model's behavior.
But what if you are a software engineer building products on top of LLMs? I'd argue that open models are a much better option even if you are using them as APIs. For at least 3 reasons:
1) The most obvious reason is reliability of your product. Relying on a closed API means that your product has a single point-of-failure. On the other hand, there are at least 7 different API providers that offer Llama3 70B already. As well as libraries that abstract on top of these API providers so that you can make a single request that goes to different API providers depending on availability / latency.
2) Another benefit is eventual consistency going local. If your product takes off, it will be more economical and lower latency to have a dedicated inference endpoint running on your VPC than to call external APIs. If you've started with an open-source model, you can always deploy the same model locally. You don't need to modify prompts or change any surrounding logic to get consistent behavior. Minimize your technical debt from the beginning.
3) Finally, open models give you much more flexibility. Even if you keep using APIs, you might want to tradeoff latency vs. cost, or use APIs that support batches of inputs, etc. Because different API providers have different infrastructure, you can use the API provider that makes the most sense for your product -- or you can even use multiple API providers for different users (free vs. paid) or different parts of your product (priority features vs. nice-to-haves)
Post
2837
Introducing the Gradio API Recorder 🪄
Every Gradio app now includes an API recorder that lets you reconstruct your interaction in a Gradio app as code using the Python or JS clients! Our goal is to make Gradio the easiest way to build ML APIs, not just UIs 🔥
Every Gradio app now includes an API recorder that lets you reconstruct your interaction in a Gradio app as code using the Python or JS clients! Our goal is to make Gradio the easiest way to build ML APIs, not just UIs 🔥
Collections
1
Papers
2
spaces
272
models
5
datasets
42
abidlabs/crowdsourced-speech-demo2
Viewer
•
Updated
abidlabs/chatinterface_with_image_csv3
Viewer
•
Updated
•
1
abidlabs/chatinterface_with_image_csv4
Viewer
•
Updated
abidlabs/chatinterface_with_image_csv2
Viewer
•
Updated
abidlabs/dddddddddd
Viewer
•
Updated
abidlabs/cccccc
Viewer
•
Updated
abidlabs/chatinterface_with_image_csv
Viewer
•
Updated
abidlabs/test7861
Updated
abidlabs/testing-text-image2
Viewer
•
Updated
•
1
abidlabs/testing-text-image
Viewer
•
Updated
•
1