Dynafed - The Dynamic Federation project

The Dynamic Federations system allows to expose via HTTP and WebDAV a very fast dynamic name space, built on the fly by merging and caching (in memory) metadata items taken from a number of (remote) endpoints.

One of its fundamental features is to be redirect GET/PUT requests to the site or cluster hosting the requested file that is closer to the client that requested it. The focus is on performance, scalability and realtime fault resilience with respect to sites that can go offline.

From the perspective of a normal user, HTTP and WebDAV clients can browse the Dynamic Federation as if it were a unique partially cached name space, which is able to redirect them to the right host when they ask for a file replica. Dynafed also supports writing.

 

The typical purpose is to present a huge distributed repository as if it were one, without necessarily keeping an always up-to-date index of all the files it contains. 

On the endpoints it natively supports HTTP, WebDAV, S3 and MS Azure. In the S3 and Azure case, the system fully supports managing hierarchical content and applying a uniform authorization scheme to any number of buckets from different providers. The authorization scheme can be based on simple rules or on more sophisticated inline scripting.

The preferred data management choice for Dynafed is to work with algorithmic, prefix-based file path translations across sites. This gives the best flexibility to the data management model, together with very quick user interaction and high performance. If needed, Dynafed can also contact an external service of file name/path translation.

 

 

Image: 

You are here