Main Page | Recent changes | Edit this page | Version history |

Printable version

 38.107.191.106 (Talk)
Log in | Help
 

Freenet Explained by Mihi

OpenFacts, the Open Source Knowledge Database

Freenet Explained (mihi)

Introduction

This page is intended to give a more thorough explanation about Freenet.

Table of contents:

Part 1: For users

CHK (Content Hash Key) KSK (Keyword Signed Key) SSK (Signed/SVK Subspace Key) SVK (Signature Verifying/Verification Key) MSK (MapSpace Key) (obsolete) KHK (Keyword Hash Key) (obsolete) ARK ( Address Resolution Key) (internal use) TUK (Time Updatable Key) (does not exist yet)

mapfiles/manifests:

Route not found Data not found Bad key Unknown Network Error Key collisions

Part 2: For developers

Part 3: Freenet Internals

Part 1: For users

Freenet basics Freenet is designed to be anonymous and totally peer to peer. Towards that end, you run a "node" that talks to other nodes. If you want some data on the network, you ask other nodes you know about if they have it. If they don't, they ask other nodes, etc. When a node replies that it does, indeed have the file, it starts passing it back up the chain. As the file travels through Freenet, nodes it passes through might keep a local copy, thereby improving future request times for the same data. This is the peer-to-peer part of Freenet. The anonymity exists because it is impossible to tell if a request for certain data came from your particular node. For example, you might just be handling it for another node which asked you for the data. Since no one can prove you asked for (or uploaded) the data, no one can punish you for your speech.

When you insert files into Freenet, your node tells other nodes that it has some content they might like. These other nodes take the file and pass it along a few times. In this way, even if your or another's node shuts down, the content will still be available on Freenet. You can never remove or update the file; the only way it will leave Freenet is if all nodes that contain the file get rid of it (usually if it is less popular than all the other files the node is holding). This is a nuisance sometimes, but is really one of Freenet's strengths: censorship is nigh impossible.

This is a major thrust of Freenet: free speech. No content on Freenet can be willfully removed or updated. Thus it is impossible to censor Freenet content by removing it or editing it unless you manage to destroy all the nodes that have the content in their datastore (and even merely requesting the file increases its propagation in the network). The Freenet community decides what is worthy speech because less popular content will eventually "fall out" of the network while popular content propagates throughout.

Freenet nodes over time specialize in certain types of keys (for example, keys starting with 'a') and learn who to go to for certain other types. Thus, Freenet has some basic routing capabilities to make requests faster. This is one reason that a long-running node will be more responsive and successful when requesting data.

Freenet nodes have a build number as part of their version number. Each node also has a last-good-build number built in. This is used in Freenet networking by refusing requests from nodes older than the last-good-build. Last-good- builds are only updated when a serious bug or network incompatibility has been introduced. In this way, node operators are forced to upgrade to newer code (a la Microsoft Word formats) because no one will talk to you if you are obsolete. I recommend you regularly update your node's code to keep current, since Freenet is very much in development.

HTL (Hops to live) A problem that arises from that structure is, that if some data is not available on Freenet, the requests would go around endlessly. To prevent this, every insert or request has a HTL number assigned. The maximum HTL at the moment is 25 (every higher HTLs will be reduced to 25). Every node a request passes usually decreased the HTL by one (and sometimes it does not, to prevent traffic analysis). So a request with HTL 25 takes longer than a reques with HTL 15 (the default) - but if HTL 15 does not yield a result, you might try again with a higher HTL (actually, if your browser supports meta refreshes - most current browsers do - a DataNotFound will be redirected to a retry with higher HTL in 20 seconds.). Inserting or retrieving with HTL 0 will affact only your local datastore. But be careful - if you insert a file with HTL 0, every try to insert it with a higher HTL will fail, as the data already is in Freenet (even if it's only in your node and that one is transient, so no one except you can get it!) Due to that problem (the data is only in your node, but nowhere else - that can happen if you insert a freesite and no one requests it, but your node's datastore is big enough to keep it) most "professional" Freesite insertion tools (FIW, fishtools) allow to bypass the local data store when inserting. If you run a transient node (like me), I'd suggest to switch that option on.

Machine readable data Due to the structure of freenet (encryption everywhere) it has to handle lots of binary data that is unreadable for humans. When this data is shown to humans (e.g. in keys), it is encoded in a way called (modified) BASE64 so that it only uses uppercase letters lowercase letters digits "-" and "~" So the most important point for users here is that these "texts" are case sensitive. So don't mess up the case of them, if you ever have to use them... Key pairs It is important to distingish "Keys" and "key pairs" on Freenet. The former is used to address content in Freenet (see next section), the latter is used to check authenticity of data that is in Freenet. Most utilities allow you to generate a key pair, if you need one. However fproxy does not have a nice form for that purpose. So if you need a key pair (e.g. for some tests with SSK keys), you have to open a terminal window or command prompt, change to your Freenet directory and type java -cp freenet.jar freenet.client.cli.Main

genkeys The result will show you a public and a private key, like this: Priv. key: AK3AmGr3TtoJYK8pH8TapcHgL7p9 Pub. key: zqAWHHcbu2KtCRbOrnZMJ7loajo Entropy: QjfR87GSJipA4JW0X4ly0g These keys are valid keys - however, do not use them for anything besides tests; as the private key is available to everyone here, this key pair has lost all of its security now.

Keys Keys are used in Freenet to store or retrieve information. Information on Freenet consists of data and metadata. The data is what the user sees from a key, the metadata contains information needed for the retriever to use the content (e.g. redirects, mapfiles, or simply the type of a file). As freenet cannot be searched (yet), it is important to know the key if you want to get information. Data stored into a key cannot be removed from freenet (except if it gets dropped because no one requests it). Trying to store data into a key that is already used creates a "key collision". So data cannot be overwritten or updated either. Generally a key looks like "freenet:type@name". The freenet: prefix is optional. Key types consist of three uppercase letters of which the last on is (at least, until now), a "K". When you reference an absolute key from a freesite, you use the syntax <a href="/type@name">link</a>

CHK (Content Hash Key) Example: CHK@hdXaxkwZ9rA8-SidT0AN-bniQlgPAwI,XdCDmBuGsd-ulqbLnZ8v~w Description: A content hash key consists of two parts of machine readable data (separated by a comma) that are generated from the content of the file. As this key type is the only one that is not limited to 32K data, it is usually used to store the actual files. All other key types usually only contain metadata. As two identical files will get the same CHK, one will get a key collision if he tries to upload a file that is already in Freenet - so CHK files save space in Freenet by preventing to insert the same file several times. As for generation of the key both data and metadata are used, you can, on the one hand, reinsert a file by changing its metadata slightly. On the other hand, CHKs with metadata mostly do not collide because of the different metadata different programs use. So for insertion programs it is a good idea not to use metadata on CHKs at all (for splitfile chunks (see below) it is mandatory not to have any metadata.) [I planned to add this paragraph to part two, but as Kadmos wanted it here, I added it here...] Theroetically, it would be possible to have two files that get the same CHK (as chk length is short and files are usually longer), but you would have to "brute force" lots of files if you want to archieve this, as the way CHKs are computed can not be reversed. I myself did not hear of any case where two different files collided. Usually you won't see many CHKs, as they are usually "hidden" in a redirect or a mapfile. If you see it, there is usually a "/filename.ext" part appended to it. Although that is not officially specified, fproxy will use that file name if you try to save a CHK target to a file. Inserting: For inserting something as CHK, simply specify "CHK@" as key. Freenet will calculate the real key for you and show it to you. Advantages:
Disadvantages:
Uses:

KSK (Keyword Signed Key) Example: KSK@gpl.txt Description: A keyword signed key is simply some human-readable text that does not need to have any relation to the file that is in it. However, usually it will describe what is in the key. The problem of those keys is that there is no control - everyone can insert data to any KSK if he does not get a key collision. Even if a KSK is used in Freenet, one can try to upload another file to that KSK with some small HTL value (HTL specifies how many nodes should be searched for the data) - if that succeeds (he "hijacked" the key), it will be kind of random which content is found first by someone who requests it. So if you try to fetch the key above, you will not necessarily get the GPL (General Public License, the license Freenet uses) - more likely you will get (i. e. I also got) a picture of a naked woman or the BSD Licence - or something completely different. Due to that fact, KSKs cannot be used well to announce places where one gets the next "edition" of a site - as everyone could insert something there and it would not even be possible to determine who did that. Usually this key type will only contain metadata that redirects to the CHK with the actual content. Inserting: The key for instering and retrieving is completely the same - something inserted as KSK@example will be retrievable as KSK@example. Advantages:
Disadvantages:
Uses: nessage indexes in chat systems like Frost

SSK (Signed/SVK Subspace Key) Example: SSK@kWu5Osv~VAI3-kH7z8QIVxklv-YPAgM/fishtools/34/active.jpg SSK@TDtkQZpHTMbhGoSxWfR-ly8BTa4PAgM,phJ~IgAADJkijtDO7iAeSw/toad/1//index.html Description: A signed subspace key consists of two parts: a public key from a keypair and some human-readable text (that may contain slashes) separated by the characters "PAgM/". There may be some "entropy" between the PAgM and the /. To insert a file into a SSK, you need the private key of that keypair - for retrieving you only need the public key. So these keys allow to control a "subspace" of Freenet keys by your key pair. So you can insert a file as e.g. SSK@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM/example/1/test.txt and tell others that you'll upload the next version at SSH@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM/example/2/test.txt. A more automated way to do this are DBRs (see below) or Edition sites (although the latter use exactly that principle to tell people where they can find the next editions). If you do not give your private key away, virtually no one will be able to hijack your SSK keys. Usually this key type will only contain metadata that redirects to the CHK with the actual content. Inserting: For inserting you need your private key. A file inserted as SSK@AK3AmGr3TtoJYK8pH8TapcHgL7p9/example.txt can be retrieved at SSK@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM/example.txt - a file inserted as SSK@AK3AmGr3TtoJYK8pH8TapcHgL7p9,QjfR87GSJipA4JW0X4ly0g/example.txt can be retrieved as SSK@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM,QjfR87GSJipA4JW0X4ly0g/example.txt. Note that these two keys will collide as the entropy is only used for encryption, not for the place in the datastore (I'm using the example keypair mentioned above for this example). Authors of new freesites should use keys with entropy, as it is harder to brute force the contents in the datastore for them. Advantages:
Disadvantages:
Uses:

SVK (Signature Verifying/Verification Key) Example: Difficult to find in Freenet, structure is like SVK@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM,EkEn2wH1J9Z2NSVIuYn~3Q Description: These keys are rarely used in Freenet - most likely because they don't have advantages. The first part of the key is a public key (with "PAgM"), the second one is generated from the file. However, a SVK collides as soon as the first part is used; so you can only insert one file per key pair. These keys should get a way of ebing updated some time ago (in V0.3 documentation that is mentioned), but most likely this will never happen. Inserting: Why inserting that kind of key? Okay, For inserting you need only your private key. A file inserted as SVK@AK3AmGr3TtoJYK8pH8TapcHgL7p9 can be retrieved at e. g. (depends on the contents) SVK@zqAWHHcbu2KtCRbOrnZMJ7loajoPAgM,EkEn2wH1J9Z2NSVIuYn~3Q (I'm using the example keypair mentioned above for this example). Advantages:
Disadvantages:
Uses: No SVKs to be found in Freenet. However, it is still possible to use them in 0.5/0.6

MSK (MapSpace Key) (obsolete) In Freenet 0.3 (?), the prefix MSK@ (e.g. MSK@SSK@foo//bar)was used to show that for getting a document you needed to examine a mapfile (also known as manifest). In Freenet 0.4 and 0.5, a double slash between the mapfile URI and the file name is used for the same purpose. KHK (Keyword Hash Key) (obsolete) In Freenet 0.2 (?), KHKs were used as KSK are used in later versions. KHKs only used a different internal handling, so the new handling gave the keys a different name. ARK ( Address Resolution Key) (internal use) An ARK looks like an SSK, but is used internally by one node to announce that it got another IP address. If a node is unreachable, the node that tried to contact it, fetches its next ARK to get the new IP address.

Matthew (amphibian) writes about ARKs:

Essentially, the node inserts the full node reference under an SSK every time the physical address changes. Other nodes look it up when the node is persistently unreachable. TUK (Time Updatable Key) (does not exist yet) the word TUK is used whenever someone discusses about keys that can be updated. Most likely no one knows how they should work, but everyone wants to have them ;-)

mapfiles/manifests Everyone that uses Freenes will see links that use manifests quite fast. Those links contain a double slash (look at your location bar to see one). For referencing a file in a mapfile, fproxy requests the mapfile (the part before the //) first. It contains lots of metadata - file names and CHKs - one CHK for each filename. Then fproxy looks up which file name you look for and fetches that CHK. Freenet is quite high-latency - so fetching two small keys take lots longer than fetching one larger key. Usually files for a freesite are inserted as SSK - so one SSK as redirect to a CHK for each file is a bit much. Using mapfiles, you only need one SSK to redirect to lots of CHKs. Another advantage is that fproxy can determine quite fast which files do not exist (if the freesite author made a mistake) and show a descriptive error message instead of fetching a SSK - if you want to see that message, click here.

DBR DBR means Date Based Redirect, and that is some kind of metadata that redirects to a different key (usually a SSK) every day, week, or any other (constant) time interval. Most DBRs update every day at midnight GMT. Internally, it adds the string after the first slash of the SSK. (This string looks like "3c057a80-" and is the number of seconds since 1970-01-01 in hex; this example is from Nubile and means 2001-11-29 - so you can imagine how outdated the information there is *g*.) Usually, at this location a mapfile is inserted - so you only need one DBR for your complete site.

ActiveLinks As mapfiles change very often when you use a DBR (even if the site remains the same all time), there is a high risk that the mapfile of an unknown site gets dropped. Also, it is somehow frustrating to click a link and get absolutely nothing, because the author forgot to insert the latest DBR slot. So most sites have a 95x32 pixel image (PNG or GIF) that is inside the mapfile and is used to link to a site. As every user who sees the link loads the image (and so the mapfile), on the one hand the mapfile is propagated through freenet and on the other hand you can see from the links (if you wait long enough) if there is a chance to get some content. On the oldest Freenet portal, The Freedom Engine (formerly known as Content of Nice or CoN), all sites are linked with ActiveLinks, so you can get some impression about which sites are available and which are not.

Edition sites For people that don't have a permanent internet connection (or for people that have other hobbies than Freenet, SCNR) it might be a problem to insert their site regurlarly (every day or week). So there has been established another type of freesites, that is not supported in any kind by the underlying Freenet Protocol (as long as there are no TUKs), but works on using the reader to determine when a site has changed. For that purpose ActiveLinks to "future" editions are added to the top (or bottom) of the freesite. When a new edition is available, users will see them and click them to get the current version. The disadvantage is that everyone who links to an Edition page has to change the link from time to time, so that users don't have to click onto several ActiveLinks until they have the latest edition. To make that link changes easier (and to allow automatic checks for new editions), it is now "best common practice" to end Edition manifest URLs by a slash and a decimal number that is increased for every edition - in other words, Edition URL will contain "/number//". Index sites like TFE try to get future URLs (with an increased number) from time to time to update their links.

FEC Splitfiles Generally a splitfile is some kind of metadata that contains links to CHKs which contain parts of the file. In "traditional" splitfiles, the whole file is split into parts of nearly the same length (usually all but the last one use a power of 2 as length, as these DataLengths can be handled best by Freenet), called "chunks" or "blocks". The advantages of these splitfiles is that there are smaller keys (every node has a maximum file size, usually defaulting to 1 MB) now, that they can be split between different nodes (one file must be stored by one node completely) and that it simplifys "resuming" downloads (one key can only be fetch completely, not in parts) when a network error occurs: The blocks already downloaded are in your datastore and you only have to fetch the rest of them. However, these are the "traditional" ones (you can also call them the "deprecated" ones, if you like that word better). The large disadvantage of that splitfiles is that, according to Murphy, you will always get all but the last block for a file - and so splitfiles leave a lot of "useless garbage" in the network. As a solution to this FEC encoding is used. This quite slow and mathematically complex algorithm can create check blocks - by default half the number of the data blocks - of which every subset of the original file's size can regenerate the file completely. So if your file has 10 data blocks, 5 check blocks are created for them, so that any 10 of these 15 blocks are enough to reconstruct the data. This seems to be inpossible when you hear it first, but it works.

So when you insert a FEC splitfile, encoding the check blocks will take some time (there is a faster native FEC codec available somewhere to be inserted into Fproxy; but I did not need it, so I cannot tell you how to use it), and insert will take 1.5 times the time of a regular splitfile. However, when someone receives the file, it is more likely that he will get the full file - and if there were unretrievable blocks, fproxy will reinsert them to "heal" the FEC splitfile. So, if you want to be good for freenet, do not disable healing in a FEC splitfile download page.

Downloading files One point may be confusing about downloading files from Freenet. Usually on the Web, you right-click the link and say "Save location to..." to save the link's destination. However, in Freenet files that are no HTML files will have an additional confirmation page where you have to confirm that you really want to download the file (as some browsers might run it and it could compromise your anonymity). So better use left-clicks there, and select "Force save to disk" on the confirmation page. Another reason why to use left-clicks is, that if you try to download a FEC split file, you can set several parameters for that on an extra page. If you chose to save to disk, you would save that page instead of the real file. If you use a download manager, disable it for freenet. Freenet files cannot be resumed directly, so there is no need for an external download manager.

NIM (Nearly Instant Messaging) NIMs seems to be the preferred way to provide feedback to a freesite author. Someone who want to send some feedback inserts it under a KSK like KSK@fxcomment-1, if that one is used, he tries KSK@fxcomment-2, and so on. The freesite author usually checks the next few number for new NIMs every few days/weeks/months and so gets the feedback. To simplify that kind of inserts, most freesites that accept NIMs provide a NIM form (like the one below). If you want to send a nim, you click some of the number links at the top until you get one that says Data not found - then you adjust the number in the text box (that one after KSK@fxcomment-), write your own text into the text box and submit. Usually, all NIM forms should work the same way - however, there was a change to Fproxy's insert mechanism some time ago, and so you might find NIM forms on really old sites that don't work as exspected. In that case, simply use another form (e.g. the one on TFE) instead - don't forget to specify the correct key!

Error messages:

The reason for that is that the request should have been sent to another node (according to the current HTL), but your node did not know another one. Fproxy also shows you some statistics what the other nodes did: if all were "unreachable", most likely there is a problem with your internet connection or your node reference table or the network is in a bad shape. Try to decrease the HTL down to 1. If you still get a RouteNotFound, the problem is your node. Check that your internet connection is working - if it is, download new seed nodes (in Windows, right click the bunny and select configure; then click "Import new node ref" and "download references") Another thing you can try is downloading the GPL - this key is one of the most requested ones and so you should be able to get it.

On the Nubile freesite there are some alternative sources for getting seed nodes, but they are either outdated or not working any longer.

In that case, enough nodes were found - but none had the data. If the data is really new (e.g. you heard on IIP that it has been inserted in the last minutes) wait a few minutes and try again - maybe it is not stored properly yet. Otherwise you can try to increase the HTL (up to 25). If the data is there, you should get it then. If you have to pay for your inet connection per minute and need some data you don't get,in times of high network load (e.g. when there was a new release) hit might help to start fetching it with HTL 25, and then diconnect your internet connection after about half a minute. Come again in half an hour and most likely you will have it in one minute or less (the data is propagated to the node next to you, as your node could not be connected)

You will get that message if you specify an incorrect URI (e.g. use wrong characters in the CHK.). Check that you did not make a typo/ did not forget to copy a part of the URI. Unknown Network Error If you see this message, please do what it says (copy the full message text and send it together with information about your system (operating system, java version) to devl@freenetproject.org. Most likely this is some kind of recursive redirects (e.g. KSK@foo redirects to KSK@bar, which redirects to KSK@foo) - or it is a serious bug in the Freenet software.

This is something you might get after an insert. If the insert was at a CHK@, this means that the file was already available on Freenet - so usually no problem. If it was another key, it might have been collided with different data. To check this, download the key, and if you get another file, you should try inserting at a different key. If you got the same file, it seems to have been collided with itself, so the file still is somewhere in Freenet (perhaps only on your own node). A rather "hackish" way to reinsert text or html files that produce a key collision is to add a newline to the file (thus making it different) and reinserting it. This should not be necessary, but sometimes it seems to be.

Client Software for Freenet For current Freenet software, check the Applications page on the Freenet website and some of the other newbie sites here in Freenet. Part 2: For developers You may check out the developer docs on the Freenet website. Those got quite good now, so I don't see much sense in writing a lot here.
Metadata Something about metadata: Every key in Freenet has metadata attached (maybe empty one though). So when you want to write freenet tools, you have to know how metadata looks like. General syntax Generally, metadata looks like Header1 Name1=Value1 Name2=Value2 ... EndPart Header2 Name1=Value1 ... EndPart Header2 Name1=Value1 ... End

The order of the headers and of the values within the headers does not matter at all. One header may appear multiple times with different attributes. All data after the final End (and the newline after it) are discarded. Usually, non-empty metadata contains a Version block with Revision=1 as first header. All the other headers are usally Document headers. The Version block can contain Encoding=gzip as well, which means that the rest of the metadata is gzipped, but I have never seen that "in the wild". All numbers are in hex.

Examples Name: Example: Usage: empty metadata 

 for splitfile segments or sometimes for files redirected to by a redirect or a mapfile.
Minimal metadata Version Revision=1 End 
 files redirected to by a redirect or mapfile.
Content type/charset only Version Revision=1 EndPart Document Info.Format=text/html;charset=utf-8 End 
 For "standalone" files and for containers. in case of text/html you can specify a character set that should be used by the anonymity filter and will be reported to the browser.
Redirect Version Revision=1 EndPart Document Info.Format=text/html Redirect.Target=freenet:KSK@gpl.txt End 

 Used for "standalone files" that redirect to a key with minimal or no metadata. Usually the target is a CHK key. The content type might be omitted here and instead placed in the metadata of the target. In case of a container, the content type application/zip may only be mentioned in the metadata of the container file itself, not in any redirects.
DateRedirect Version Revision=1 EndPart Document Info.Format=text/html DateRedirect.Increment=15180 DateRedirect.Offset=0 DateRedirect.Target=freenet:KSK@gpl.txt End 

 Used for DBR sites - usually redirects to a SSK key with a manifest in it, but KSK is also possible (but hijackable). The increment and offset can be left out, if they are default values (as shown in this example, all values in seconds.)
Manifest file/mapfile Version Revision=1 EndPart Document Name=index.html Info.Format=text/html Redirect.Target=freenet:CHK@foo EndPart Document Name=image.png Info.Format=text/html Redirect.Target=freenet:CHK@bar EndPart Document Name=image.png Info.Format=text/html Redirect.Target=freenet:CHK@foo End 

 Used for freesites in general. To access a site in here, add //name to the end of the freenet key. If only // is added, the default key (the one without Name) is fetched. The order of the attributes within a section can be used to "guess" the insertion tool that inserted this site.
Splitfile Version Revision=1 EndPart Document Info.Description=file Info.Format=application/zip Info.Checksum=15a281c3382529a50eafe526cdb0ceae60012e21 SplitFile.AlgoName=OnionFEC_a_1_2 SplitFile.Size=8ebd2 SplitFile.BlockCount=6 SplitFile.CheckBlockCount=3 SplitFile.Block.1=freenet:CHK@... SplitFile.Block.2=freenet:CHK@... SplitFile.Block.3=freenet:CHK@... SplitFile.Block.4=freenet:CHK@... SplitFile.Block.5=freenet:CHK@... SplitFile.Block.6=freenet:CHK@... SplitFile.CheckBlock.3=freenet:CHK@... SplitFile.CheckBlock.2=freenet:CHK@... SplitFile.CheckBlock.1=freenet:CHK@... End 

 Used for FEC splitfiles. The checksum is SHA1, all numbers (the block and check block numbers as well) are in hex as usual.

Part 3: Freenet Internals

Keys The following text is not by me. However I think despite the age of it, it might still be useful. (I tried to strip out parts I am sure of that they have changed). If someone tells me what is still true and what isn't, i might add that in the next edition of this freesite. Bestiary of Keys: by Michael Wiktowy Last Edited: 22 August 2000 I would like to bring the discussion of all these keytypes under one roof. It appears that there is a hierarchy of keytypes forming and I would summarize it here for clarity for myself and hopefully for others. Many of the keys are first included in Freenet Reference Node v.0.3.

Mini Glossary: Purpose - the reason for this type of key to exist Generation - the steps and information required to create the key Reference - the URI to be imbedded in a webpage, redirector, etc. for a request Search Key - Information that is supplied to the node in order to route the request/insert. This is the reference that is kept in the node's store. Encryption Key - the password used to encrypt the main data Contents - Node-readable Metadata - the info that must be included in plain-text form to each node Contents - Data - the contents of the encrypted data Routing - the types of checks that are performed as the document gets delivered from node to node on insert and request Comments - useful comments about the key extracted from the mailing lists

Main Keytypes:

KHK (keyword hash key) (*deprecated in 0.3 - superceeded by KSK below*)

[snip]

KSK (Keyword Signed Key) Purpose: Replacement of KHKs with a secure, untamperable but still guessable, human readable keyword derived key. This key is a meld of the KHK and SVK concept.

This concept offers protection against hash LSB incrementing attacks by proving that the requestor knew the unhashed keyword. It also protects against nodes undetectably changing the data contents under the key on insert or reply by signing the contents. The node needs to know the unhashed keyword in order to forge the reply or insert data so it provides a minimal amount of security whereas the old KHK provided none.

Generation: Private Key = k = hash(Document Name) Public Key = y = g^k mod p Digitally sign encrypted data using k

Reference: free:KSK@Document Name

Search Key: hash(public key)

Encryption Key: Document Name

Contents: 

    Node-readable Metadata:
The KSK will contain the same metadata as the KHK in addition to the public key and the signature for integrity verification. 
    Data:
The only data will be the redirector to a SVK or CHK

Routing: KSK routing is the same as KHKs but routing nodes can check the integrity of the document in transit.

Comments: From posting by JJ Furman <jfurman@excitehome.net>: It involves an alternate way to calculate the hash used to refer to the document by generating Public/Private pairs. The steps below would be required each time a document was inserted:

a. First calculate the hash of the key (readable name) and put it into H. This is not really a necessary step but may help the crypto strength by scattering.

b. Next seed a deterministic random number generator with H and use it to find suitable primes to be used for a public/private key pair. Note that this algorithm should be designed to always output the same two primes based on H.

c. Now, create a public/private key pair using the above gotten primes.

d. Add the public key to the contents you are putting at this KHK as a piece of meta data. Sign the contents with the private key. This is just like the SVK and should sound really familiar.

e. put the data into freenet under the hash of the public key.

Now, when someone wants the data they can go through the above steps, only instead of putting the data, they request it. It can be verified along the way using exactly the same method of SVK, probably wouldn't even need to change the code.

From posting by Hal <hal@finney.org>: This is a good idea; a couple of comments.

First, if we use DSS signatures there is an easy way to generate a public/private key pair given a hash. You start with a readable_name (like /text/books/bible), then set the private key to the hash, k = hash(readable_name). The public key is then y = g^k mod p.

We then basically insert like a SVK using y as the public key. Conceptually, y is the Freenet routing key. (Actually we use hash(y) as the routing key and include the full value of y in the metadata; this is equivalent but uses a shorter routing key.)

The security this provides is as follows:

It is like a KHK, but includes a proof that the original creator of this data packet knew a readable_name which hashes to the routing key.

That's it.

SVK (Signature Verification Key) Purpose: The primary purpose of SVKs is to provide a means of updating data. The SVK will also be the main index of SVKs and CHKs. These will not be guessable. They must be unique. SVKs are tamperfree since a user needs to have the private key associated with the SVK in order to change it and any changes due to a cancer node will just generate a different SVK or an unreadable one.

Generation: Generate of Private/Public cryptographic key-pair Take SHA1 hash of Public Key Digitally sign encrypted data using private key

Reference: freenet:SVK@hash(public key),Encryption Key

Search Key: hash(public key)

Encryption Key: User entered keyword or randomly generated sequence to enhance entropy

Contents: 

    Node-readable Metadata:
The public key *and* the private key signature of the encrypted data must be included in order for the nodes to verify the integrity of the SVK. The version number associated with the SVK should always be included so that nodes can resolve version conflicts between identical SVKs. The SVK will *never* contain it's own encryption key. 
    Data:
Since this document is an index it will contain a list of keys and a possible descriptor for each key. The analogy for this key would be a html document on the www.

Routing: SVK routing will be the same as now on insert and request. An additional SVK collision mechanism would be reqired such that if a node had two references to the same SVK, the signature would be verified to be the same and the older SVK data would be discarded. Having newer SVKs reach and update all older is currently a problem. "Following through" requests for SVKs will create congestion at focus nodes for that particular keyspace. Following through on inserts will not reach a significant portion of nodes. A potential option is to create bidirectional reference trees so that the freenet as a whole can determine where all occurances of a particular key are but this may be difficult to establish without revealing the structure of freenet to malicious parties. This option has the most potential though.

CHK (Content Hash Key) Purpose: This key contains the meat of the data on freenet. It will carry all the binary data building blocks for the content to be delivered to the client for reassembly and decryption. The CHK is unique by nature and provides tamperproof content. A cancer node messing with the data under a CHK will immediately be detected by the next node or the client. CHKs also reduce the redundancy of data since the same data will have the same CHK and will collide on insert.

Generation: Take SHA1 hash of encrypted data

Reference: freenet:CHK@hash(encrypted data),Encryption Key

Search Key: hash(encrypted data)

Encryption Key: User entered keyword or randomly generated sequence to enhance entropy

Contents: 

    Node-readable Metadata:
CHKs will contain an absolute minimum of metadata. It will *never* contain it's own encryption key. 
    Data:
CHKs will contain the encrypted data to be supplied to the client for client-side decryption using an encryption key obtained from the reference that pointed to it.

Routing: CHK routing would be handled exactly as routing is handled now. The only exception would be that CHK checks would be incorporated on insert and request to ensure the integrity of the encapsulated data.

SSK (SVK Subspace Key)

Purpose: The primary purpose of SSKs is to provide a means of defining a namespace for forums and such.

Generation: Generate Private/Public cryptographic key-pair Create Document Name Digitally sign hash(Document Name)+encrypted data using private key Take SHA1 hash of public key to create SVK Take SHA1 hash of SVK+hash(Document Name)

Reference: freenet:SSK@Document Name,Parent SVK

Search Key: hash(hash(public key) + hash(Document Name))

Encryption Key: Document Name

Contents: 

    Node-readable Metadata:
Must include Public Key, Document ID and Signature so the node can verify the integrity 
    Data:
Same as SVKs

Routing: SSK routing will be the same as SVK.

Comments: (From posting by "Scott G. Miller" <scgmille@indiana.edu>)

SSK's are simply the client-side representation of SVK's with a document name. What these allow you to do is create a very simple subspace in Freenet with guessable keys but control over insertion.


[Main Page]

Main Page
Recent changes
Random page
Questions and suggestions
Current events

Edit this page
Discuss this page
Version history
What links here
Related changes
Special pages
OpenFacts bug reports