Launching Sovereign Chat 0.1

Because digital Privacy is often complicated... Launching FOSS AI Chatbot that answers all questions about online Privacy & Security

SOLIDUS'S INSIGHTS

4/4/20242 min read

After like three afternoons of researching and trying to find how to build something I wanted to build long time ago, I got it (almost).

Because my knowledge of coding is almost non-existent (OK I've done some C# but you know learning anything new is pain), it had to be simple.

What I used?

I found this nice framework: Streamlit+ Embedchain + Huggingface API

The most important part is Embedchain, that allows me to use my own docs, websites, videos and more.

Which data I used?

I imported data in forms of URLs that are embedded into LLM. Right now, while I'm testing multiple LLMs, I'm only using small dataset so the embedding doesn't take long. You can take a look at the bigger dataset.

Problem with AI API and running models locally

Right now, I'm facing problems with LLMs. Only functioning AI API was the one from OpenAI, but I don't want to use that one because of data collecting, censorship and overall OpenAI policy. Sow we used HuggingFace API instead.

The issue

After running embedding without problems you can ask the question, but you won't get answer and this error will show in Terminal.

I tried re-installing all the packages, even different OSs and running in clean VMs but still got the same error.

Last resort was GPT4LL, but on main PC I got error that BaseLlmConfig couldn't be imported. In the VM no error was reported, but I also got no answer (maybe not enough computing resources???)

How can you help?

I would like you to try it out yourself so I can figure out where the bug is. To change which LLM is used, just change the config.yaml file. The least issues I had with Mistral API and GPT4ALL (that runs locally).

If the HuggingFace doesn't work, try out GPT4ALL. If it runs smoothly, it would be the nice and maybe better than using APIs. You can see the source code here.

I would also appreciate if you submitted any favourite resources for online privacy and security. Right now, we are featuring:

@privacy_guides

@sethforprivacy

@bisq_network

@Bitwarden

@BitcoinQ_A

@econoalchemist

@SamouraiWallet

@ResPrivacy

You can subscribe to my newsletter to get updates or follow me on Twitter.

I believe we can figure this out and help others to understand online Privacy & Security online more.

Thank you.

And be Sovereign. Not only online.

Marconius Solidus

Subscribe to my newsletter

You'll also receive FREE E-book, where you will learn how to protect your email against data leaks, spam and unwanted newsletters.