Follow

There's a misconception that you can "store things on the blockchain", let alone perminantly so. No: blockchains just gets everyone to agree on the same order of events, the same "merkle-tree".

To store data peer-to-peer you need a Distributed HashTable. And to store data perminantly, just save it to your harddrive! Seriously we don't have better solution, and I've heard it argued nor should we.

This morning I will describe how DHTs work.

A normal hashtable maps keys to values by converting the key's "hash" into an index into an array (a.k.a. RAM segment), with fallback logic in case of a "collision".

A Distributed HashTable spreads this mapping from keys to values accross multiple "peers", each of which are given an ID comparable to the hashes being looked up. And it works by establishing an invariant that each peer knows the entries and other peers closest to it's ID, so they can always point you closer to the data you want.

Using that invariant you can easily find the peer responsible for storing a particular key by asking a a small fraction (log n) of all peers, which you can then ask to set or get the data. Though usually there's some duplication to increase reliability and indirection to spread out network load.

To join a DHT you start from a centralized "bootstrap" list peers, before querying it for your own randomly chosen ID to establish that invariant. And to see if that ID's already taken.

As for existing applications of DHTs, they underly:

* Tor Hidden Services
* Bittorrent trackers
* IPFS, FreeNet, Dat, etc
* Looking up peers in Tox
* Any others?

@alcinnz actually you do store things on the blockchain, but they're typically just really small.

@jasper Sure, if you can be sure everyone's storing the full merkle-dag. Which as those blockchains grow won't be reality.

And blockchains don't provide a means to look up data not already on the client.

@alcinnz the point of a lightweight client should be able to get any part of the blockchain? (And use Merkle proofs to verify that they are infact in there.)

Infact you'd get the DHT address that way, if some storage is relevant.

That said, i kindah think it might be better to just do it the direct way. For example sooner replace Uber with just a co-operative with servers somewhere. If the police wants to stop it, they'll just steal the cars, and the blockchain has tons of unknown unknowns..

@jasper I'm a bit confused about the concept of "getting the DHT address", because my understand is that it's computed directly from the key being looked up. Just like what happens for local HashTables.

But vice versa you could use a DHT to let those lightweight clients fetch any part of the blockchain's merkle-dag peer-to-peer.

@alcinnz as i understand it, it can find.. a thing based on for instance sha256. The checksum can be used as address. In the case of Tox it's the hash of the public key, and then you find the ip of that public key. (i think..) (or Tor, i presume, the IP .. of the exit .. of the site.) But i could easily be off..

The hash can be used as address. And in case of sha256 it is 64 bytes, small enough to leave on a blockchain, for instance to "refer to it's website".

@alcinnz my impression is that Ethereum .. was trying to do this, i have no idea if they succeeded.

I really should revisit... But i am still under the impression that it might not be such a great idea after all.

@jasper Ah yeah, I think I can see your thinking now!

A blockchain can aid in assigning names to some more decentralized identifier, that's been tried a few times and does make sense. Because having everyone agree on which order the names were claimed does help in resolving conflicts.

@alcinnz like namecoin. Ethereum contracts can do it.. dunno if seriously used.

Rules regarding what addresses/public keys names have can be arbitrary. The "website" text should match the contract. I mean, might want to do it in a way to minimize surprised.

@alcinnz Inasmuch as a blockchain consists of its blocks (the entire chain of them), you can store things in the blocks, and thus you can store things on the blockchain. As long as there are full nodes storing the entire blockchain, those things are permanently stored.

But if users store a pruned blockchain and you define the blockchain just by its merkle tree, then yeah.

Sign in to participate in the conversation
FLOSS.social

For people who care about, support, or build Free, Libre, and Open Source Software (FLOSS).