When models are hitting new highs on benchmarks, are they good in general or good at getting good results on benchmarks? Björn Mattsson and Pat Walter's preprint on data leakage is available on biorxiv:

bioRxiv

Identifying and Addressing Systematic Data Leakage in Protein-Ligand Affinity Benchmarks

Accurate prediction of protein-ligand binding affinity is a crucial goal in structure-based drug discovery, with the potential to significantly sho...

#bio #ai

Soon the question won't be a model's cost or "accuracy", it'll be who made it. The lab behind a model shapes how it nudges and persuades. We already see that people have "a high tendency to follow the potentially harmful medical advice" from model output (1) - imagine the power these providers can have, not just in intelligence but in persuasion. 1:

MIT Media Lab

People Overtrust AI-Generated Medical Advice despite Low Accuracy – MIT Media Lab

We conducted a study in which a total of 300 participants gave evaluations for medical responses that were either written by a medical doctor on an...

nostr is an amazing platform to build on! View quoted note →

The greatest things to come from the AI boom is that people will meet each other more and, ironically, since we have an explosion in new apps and websites being built these days, will spend less time using them.

Event reminders working as intended 😀

Hello

Hello @Leannehall

More: Slack has a hidden always-opt-in policy for their "global model" - opting out can only be done via email:

How To Opt Out of Slack AI

Opt out of Slack AI with this quick guide and easily stop Slack from using your personal data to train their AI models.

View quoted note →

Testing new relays

Playing around with fetching skills and executing them locally on a phone. Limited use cases now but I think at some point you could fetch a whole “business setup” instruction and have an offline thing help you start something! Here I fetched a WiFi QR maker skill from

AgentList

WiFi QR Generator for Google Edge Gallery — AgentList

WiFi QR Generator for Google Edge Gallery is an AI skill on AgentList. Creates a sharable QR code of your wifi details for guests to scan. Built fo...

and it executed the JavaScript in it based on the text. Fully offline running the model on my phone. Quite fun. Now looking for more use cases!

Data really becoming the new gold, no longer hidden by a layer of software pretending to be the gold. Now-defunct companies selling their internal data for training:

Forbes

AI’s New Training Data: Your Old Work Slacks And Emails

AI’s New Training Data: Your Old Work Slacks And Emails

I recently noticed Slack has restricted access to data downloads to a higher tier of subscription, not the first paid tier but the second one. Only public channel messages are downloadable. The same will happen for other apps, the moat will be your data, and the aim to make accessing data harder - complicated structures, training in a very custom solution that is so specialised you cannot leave. Before: Big companies tried to harvest as much data as possible Now: They need to protect it Time to be aware of the data your organisation has, and the optionality you currently have in regards to the software you use and the licenses you have with the stewards of the information!

Trying to get some thoughts out these days, still working on making it interesting, but hopefully it's interesting to someone! Wrote some thoughts about what knowledge organisations might want to think about when letting bigCorp add "smart AI" on their content for them. View article →

Really amazing that nostr came at a time when anyone can build apps at speed - suddenly we have a defined way to do all sorts of things properly, anchored in this amazing protocol. Let’s use it.

The Best Nostr App Won’t Say Nostr on the Tin

Lets make apps, not (only) nostr clients

Here’s a basic agent definition I recommend teams use as a basis for their work to increase the quality and reduce human frustration:

AgentList

Baseline Coding Agent — AgentList

Baseline Coding Agent is an AI agent on AgentList. A base for software teams building across projects, it aims to improve readability, reviewabilit...

#agents #opencode

If you rely on AI to do information (read: code) exchange within your company, it’s as crucial to tell all the implementations of it how you’d like to have this information structured. It’s a bit like company policy - if everybody knows and follows it, things get predictable and not messy!

Ok hear me out. Passkeys on your phone / google / apple sync. It seems to already provide quite a good deal of security for a seed. It works quite well to deterministically create NOSTR keys from those. No server hosting your keys, no bunker, no plugin that doesn’t work on iOS anyway, no background audio to enable remote signing, and cross browser capability. Don’t send the key - recreate it! Maybe a security issue? But it doesn’t look like! What am I missing?