Skip to content

External Document Management

Introduction

The following document contains snippets of smart contracts and sample API endpoints related to how a user can store files externally and reference those files on STRATO chains. The smart contracts presented show examples of how a user can store data about externally stored files in STRATO and how the data can be modeled to reference those files. The API endpoints show examples of endpoints that can be implemented in an application so that the user can retrieve or modify the data stored in each contract.

Assumptions

  • The user will be storing the files externally in an AWS S3 bucket

Smart Contract Snippets

ExtFileStorageDapp Contract

/**
 * Single entry point to all the project's contract
 * Deployed by the deploy script
 */
 contract ExtFileStorageDapp {

   // internal
   address owner; //  debug only
   UserManager userManager;

   constructor() {
     owner = msg.sender;  //  debug only
     userManager = new UserManager(msg.sender);  
   }
 }
This contract is the main dapp contract that is deployed in a deploy script.

Attestation Contract

Attestation of a file means that a user agrees with the contents and data of a file. A user “signs” their user address to the file’s smart contract to attest to its legitimacy. The following is an attestation smart contract that can be used in an app:

contract Attestation {
  uint public version;
  string public fileKey;
  address public attestor;
  uint public timestamp;

  constructor(address _attestor, string _fileKey, uint _version) {
    attestor = _attestor;
    fileKey = _fileKey;
    version = _version;
    timestamp = now;
  }
}

When a user attests to the validity of a file, the user’s address, the file’s key, the version of the file, and the timestamp in which the file was attested are all data contained in the contract.

External File Contract

Users can update externally stored files to a new version in STRATO. The following contract is an example of the type of data that can be stored for a version of an external file: 

import "./Attestation.sol";

contract ExternalFile {
    uint256 public version;
    string public fileKey;
    string public fileHash;
    string public comments;
    string public uri;
    uint256 public timestamp;
    address public uploadedBy;
    Attestation[] public attestations;

    constructor(
        string _fileKey,
        string _fileHash,
        string _uri,
        string _comments,
        address _uploadedBy,
        uint256 _version
    ) public {
        fileKey = _fileKey;
        fileHash = _fileHash;
        uri = _uri;
        uploadedBy = _uploadedBy;
        version = _version;
        timestamp = now;
        comments = _comments;
    }

    function attest(address _attestor) public returns (uint256) {
        attestations.push(new Attestation(_attestor, fileKey, version));
        return 1;
    }
}

This smart contract stores the version of the external file, the file key, the file hash, the URI which points to where the file is stored externally, and other related information about the file. By design, the contract does not provide any methods to modify existing data except to add a new attestor.

Membership Contract

The owner of files can share membership to a particular file with the use of a Membership contract. The following contract is an example of how such a contract can be modeled:

contract Membership {
  string public fileKey;
  string public action;
  address public member;
  uint public timestamp;

  constructor(address _member, string _action, string _fileKey) {
    member = _member;
    action = _action;
    fileKey = _fileKey;
    timestamp = now;
  }
}

File Manager Contract

The File Manager contract is used to represent a specific externally stored file. It also provides the user methods to modify existing data regarding the file. The following is an example of a File Manager contract:

import "./ExternalFile.sol";
import "./Membership.sol";

contract FileManager {
    uint256 public numVersions;
    string public fileKey;
    string public fileName;
    string public fileDescription;
    uint256 public createdAt;
    uint256 public lastActivityAt;
    address public owner;
    bool public isInitialized;
    Membership[] public memberships;
    ExternalFile[] fileVersions;

    event MemberAdded(address member, string enode);
    event MemberRemoved(address member);

    function initialize(
        string _fileKey,
        string _fileName,
        string _fileDescription,
        string _comments,
        string _fileHash,
        string _uri
    ) public {
        numVersions = 0;
        isInitialized = true;
        fileKey = _fileKey;
        fileName = _fileName;
        fileDescription = _fileDescription;
        addFileVersion(_fileHash, _comments, _uri);
        createdAt = now;
        lastActivityAt = now;
        owner = msg.sender;
    }

    function addFileVersion(
        string _fileHash,
        string _comments,
        string _uri
    ) public {
        numVersions++;
        ExternalFile file = new ExternalFile(
            fileKey,
            _fileHash,
            _uri,
            _comments,
            msg.sender,
            numVersions
        );
        fileVersions.push(file);
    }

    function transferOwnership(address newOwner, string enode)
        public
        returns (uint256)
    {
        if (msg.sender == owner) {
            uint256 status = share(newOwner, enode);
            if (status == 0) return status;
            owner = newOwner;
            lastActivityAt = now;
            return 1;
        } else {
            return 0;
        }
    }

    function share(address member, string enode) public returns (uint256) {
        emit MemberAdded(member, enode);
        memberships.push(new Membership(member, "MEMBER_ADDED", fileKey));
        if (member.balance == 0) member.transfer(1000000000000000000000);
        lastActivityAt = now;
        return 1;
    }

    function remove(address member) public returns (uint256) {
        if (msg.sender == owner && member != owner) {
            emit MemberRemoved(member);
            memberships.push(new Membership(member, "MEMBER_REMOVED", fileKey));
            lastActivityAt = now;
            return 1;
        }
        return 0;
    }

    function attest(uint256 version) public returns (uint256) {
        lastActivityAt = now;
        return fileVersions[version - 1].attest(msg.sender);
    }
}

The smart contract stores the number of versions of the particular file, the file name, the file key, all the different versions of the file, etc. Additionally, the smart contract provides a few methods to modify existing data such as adding a new version of a file, transferring ownership of a file, attesting a version of a file, etc

Sample API

This section outlines API endpoints involving files that can be implemented in an application.

POST /api/v1/files/ - Uploading a File

The request body takes the following parameters: - filePath (of the file to be uploaded) - fileDescription - comments

The deployed application should have the external storage bucket set upon deployment along with the access key ID and secret access key necessary to make calls to the bucket. The target file will be uploaded to S3, and then both the File Manager contract and the first version of the External File contract with the information of the target file will be uploaded to STRATO:

  static async uploadFile(req, res, next) {
    const { fileDescription, comments } = req.body;

    if (!req.file) {
      rest.response.status400(res, "Missing file");
    }
    try {
      const fileKey = `${moment()
        .utc()
        .valueOf()}_${req.file.originalname}`;

      const fileHash = crypto
        .createHmac("sha256", req.file.buffer)
        .digest("hex");

      const uploadResult = await uploadFileToS3(
        `${fileKey}_${fileHash}`,
        req.file.buffer,
        req.app.get(constants.s3ParamName)
      );

      const fileInterface = await fileJs.uploadFile(
        req.user,
        {
          fileKey,
          fileName: req.file.originalname,
          fileHash,
          uri: uploadResult.Location,
          fileDescription,
          comments
        },
        { config }
      );
      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }

The function creates the additional information necessary for the File Manager contract such as fileKey, fileHash, etc. Additionally, the URI is taken from the result of the S3 upload. The External File contract is created by calling the initialize function of the File Manager contract. The response should look something like this: API Response

GET /api/v1/files/:filekey - Retrieving a File

The API request takes the following parameter: - fileKey

The /:filekey in the API URL should be replaced with the filekey of the target file to be retrieved.

async function getFile(user, _fileKey, options) {
  const fileKey = `eq.${_fileKey}`;
  const searchOptions = {
    ...options,
    query: options.chainIds
      ? {
          fileKey
        }
      : {
          fileKey,
          chainId: `in.${util.toCsv((await getChains(user)).map(c => c.id))}`
        }
  };
  const fileManagerSearchResults = await rest.search(
    user,
    { name: fileManagerContractName },
    searchOptions
  );
  if (fileManagerSearchResults.length > 0) {
    const file = fileManagerSearchResults[0];

    const versionsQuery = await rest.search(
      user,
      { name: fileContractName },
      searchOptions
    );

    // get attestation for file
    const attestationsQuery = await rest.search(
      user,
      { name: "Attestation" },
      searchOptions
    );

    // get memberships for file
    const membershipsQuery = await rest.search(
      user,
      { name: "Membership" },
      searchOptions
    );

    // get users
    const users = await rest.search(user, { name: "User" }, { config });

    // get history
    const history = await rest.search(
      user,
      { name: `history@${fileManagerContractName}` },
      {
        config,
        query: { fileKey, chainId: `eq.${file.chainId}` }
      }
    );

    const owner = users.find(u => u.account == file.owner);
    const chains = await getChains(user, [file.chainId]);

    const members = [];
    if (chains.length > 0) {
      chains[0].info.members.forEach(m => {
        const member = users.find(u => u.account == m.address);
        if (member) {
          members.push({
            username: member.username,
            address: m.address
          });
        }
      });
    }

    const transferOwnerships = history.reduce((t, h, i) => {
      if (i == 0) {
        return t;
      }
      if (history[i - 1].owner != h.owner) {
        const member = users.find(u => u.account == h.owner);
        if (member) {
          t.push({
            action: "OWNERSHIP_TRANSFERRED",
            member: {
              username: member.username,
              address: member.account
            },
            timestamp: moment(new Date(h.block_timestamp)).unix()
          });
        }
      }
      return t;
    }, []);

    const attestations = attestationsQuery.map(a => {
      const attestor = users.find(u => u.account == a.attestor);
      if (attestor != undefined) {
        return {
          ...a,
          action: "VERIFICATION",
          attestor: {
            username: attestor.username,
            address: attestor.account
          }
        };
      }
      return a;
    });

    const versions = versionsQuery.map(v => {
      const uploadedBy = users.find(u => u.account == v.uploadedBy);
      if (uploadedBy) {
        return {
          ...v,
          action: "NEW_VERSION",
          attestations: v.attestations.map(a =>
            attestations.find(at => at.address == a)
          ),
          uploadedBy: {
            username: uploadedBy.username,
            address: uploadedBy.account
          }
        };
      }
      return v;
    });

    const memberships = membershipsQuery.map(m => {
      const member = users.find(u => u.account == m.member);
      if (member != undefined) {
        return {
          ...m,
          member: {
            username: member.username,
            address: member.account
          }
        };
      }
      return m;
    });

    const auditLog = [
      ...new Set([
        ...versions,
        ...memberships,
        ...attestations,
        ...transferOwnerships
      ])
    ];

    auditLog.sort((a, b) => a.timestamp - b.timestamp);

    return {
      ...file,
      members,
      owner: owner
        ? { username: owner.username, address: owner.account }
        : file.owner,
      versions,
      auditLog
    };
  } else {
    return undefined;
  }
}
The function looks for the File Manager contract associated with the given filekey. Once it is found, it gathers information about the requested file stored in the contract and returns the data such as the members with access to the file, the file owner, versions of the file, etc.

POST /api/v1/files/:filekey/versions/:version/attest - Attesting a File

The API request takes the following parameters: - filekey - version

The /:filekey in the API URL should be replaced with the filekey of the target file to be attested, and /:version in the API URL should be replaced with the version of the target file to be attested

  static async attestFile(req, res, next) {
    if (!req.file) {
      rest.response.status400(res, "Missing file");
    }
    try {
      const fileKey = req.params.fileKey;
      const fileInterface = await fileJs.bind(req.user, fileKey);

      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${fileKey}`
        });
        return;
      }

      const version = parseInt(req.params.version);

      if (!isNumber(version)) {
        res.status(400).json({
          success: false,
          data: `Invalid version "${req.params.version}". Should be a number`
        });
      }
      const fileHash = crypto
        .createHmac("sha256", req.file.buffer)
        .digest("hex");

      const fileVersion = fileInterface.file.versions.find(
        v => v.version == version
      );
      if (!fileVersion) {
        res.status(404).json({
          success: false,
          data: `Could not locate file version "${req.params.version}"`
        });
      }
      if (fileVersion.fileHash != fileHash) {
        rest.response.status400(
          res,
          "File hashes do not match. Attestation failed."
        );
        return;
      }
      const result = await fileInterface.attest(version);
      if (result == "0") {
        rest.response.status500(res, "Unknown issue while attesting file");
      }
      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }
The function searches for the FileManager contract with the matching fileKey and then searches for the version matching the number provided. Once found, the attest function of the FileManager contract is called.

Before Attestation API call: Before Attestation API Call

After Attestation API call: After Attestation API Call

POST /api/v1/files/:filekey/members - Sharing a File

The API request takes the following parameters: - filekey (replaced in the API URL) - member (address of user to share a file with included in the request body) 

  static async share(req, res, next) {
    try {
      const fileKey = req.params.fileKey;
      const fileInterface = await fileJs.bind(req.user, fileKey);

      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${fileKey}`
        });
        return next();
      }

      if (!req.body.member || !util.isAddress(req.body.member)) {
        res.status(400).json({
          success: false,
          data: `Missing or invalid member address`
        });
      }

      const result = await fileInterface.share(req.body.member);
      if (result == "0") {
        res.status(400).json({
          success: false,
          data: `Invalid member or user does not have permission`
        });
      }

      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }

The function looks for the File Manager contract associated with the given filekey; then it checks to ensure that the target account to share the file with is not already a member. It then calls the share function of the File Manager contract to share the contract with the target account.

GET /api/v1/files/:filekey/versions/:version - Downloading a File

The API request takes the following parameters: - filekey - version The /:filekey in the API URL should be replaced with the filekey of the target file to be downloaded, and /:version in the API URL should be replaced with the version of the target file to be downloaded.

  static async downloadFile(req, res, next) {
    try {
      const fileInterface = await fileJs.bind(req.user, req.params.fileKey);
      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${req.params.fileKey}`
        });
        return;
      }

      const version = parseInt(req.params.version);

      if (!isNumber(version)) {
        res.status(400).json({
          success: false,
          data: `Invalid version "${req.params.version}". Should be a number`
        });
      }

      const fileVersion = fileInterface.file.versions.find(
        v => v.version == version
      );

      if (!fileVersion) {
        res.status(404).json({
          success: false,
          data: `Could not locate file version "${req.params.version}"`
        });
      }

      const fs = getFileStreamFromS3(
        `${fileInterface.file.fileKey}_${fileVersion.fileHash}`,
        req.app.get(constants.s3ParamName)
      );
      res.attachment(
        `${basename(
          fileInterface.file.fileName,
          extname(fileInterface.file.fileName)
        )}_${version}${extname(fileInterface.file.fileName)}`
      );
      fs.pipe(res);
    } catch (e) {
      next(e);
    }
  }

The function looks for a file that has the matching filekey and version number as requested in the API call. Once it is found, a request to S3 is made to get the file to be given to the user as a downloadable file. The response includes a content-disposition header which indicates that the file transmitted should be able to be downloaded.

POST /api/v1/files/:filekey/transfer - Transferring Ownership of a File

The API request takes the following parameters: - filekey (replaced in the API URL) - member (address of user to transfer file ownership to included in the request body)

  static async transfer(req, res, next) {
    try {
      const fileKey = req.params.fileKey;
      const fileInterface = await fileJs.bind(req.user, fileKey);

      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${fileKey}`
        });
        return next();
      }

      if (!req.body.member || !util.isAddress(req.body.member)) {
        res.status(400).json({
          success: false,
          data: `Missing or invalid member address`
        });
        return next();
      }

      const result = await fileInterface.transferOwnership(req.body.member);
      if (result == "0") {
        res.status(400).json({
          success: false,
          data: `Invalid member or user does not have permission`
        });
        return next();
      }

      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }

The function looks for the File Manager contract associated with the given filekey. Once found, the transfer function from the File Manager contract is called to transfer the ownership of the file to the target account.

DELETE /api/v1/files/:filekey/members - Remove Membership from a File

The API request takes the following parameters: - filekey (replaced in the API URL) - member (address of user to share a file with included in the request body) In this sample scenario, the ownership of the file has been transferred to a different user, and the original owner’s membership to the file is being removed.

  static async remove(req, res, next) {
    try {
      const fileKey = req.params.fileKey;
      const fileInterface = await fileJs.bind(req.user, fileKey);

      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${fileKey}`
        });
        return next();
      }

      if (!req.body.member || !util.isAddress(req.body.member)) {
        res.status(400).json({
          success: false,
          data: `Missing or invalid member address`
        });
        return next();
      }

      const result = await fileInterface.remove(req.body.member);
      if (result == "0") {
        res.status(400).json({
          success: false,
          data: `Invalid member or user does not have permission`
        });
        return next();
      }

      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }

The function looks for the File Manager contract associated with the given filekey. Once found, the remove function from the File Manager contract is called to remove membership of the file from the target account.

POST /api/v1/files/:filekey/versions - Upload New Version of a File

The request body takes the following parameters: - filekey of the file to be added a new version to - filePath (of the file to be uploaded) - fileDescription - comments

The deployed application should have the external storage bucket set upon deployment along with the access key ID and secret access key necessary to make calls to the bucket. 

 static async uploadVersion(req, res, next) {
    if (!req.file) {
      rest.response.status400(res, "Missing file");
    }
    try {
      const fileKey = req.params.fileKey;

      const fileInterface = await fileJs.bind(req.user, fileKey);

      if (!fileInterface) {
        res.status(404).json({
          success: false,
          data: `Unable to locate file with key ${fileKey}`
        });
        return;
      }

      const fileHash = crypto
        .createHmac("sha256", req.file.buffer)
        .digest("hex");

      const uploadResult = await uploadFileToS3(
        `${fileKey}_${fileHash}`,
        req.file.buffer,
        req.app.get(constants.s3ParamName)
      );

      const { comments } = req.body;


      await fileInterface.addFileVersion({
        uri: uploadResult.Location,
        fileHash,
        comments
      });
      const file = await fileInterface.getFile();
      rest.response.status200(res, file);
    } catch (e) {
      next(e);
    }
  }

The function looks for the File Manager contract associated with the given filekey. Then it uploads the new version of the file to S3, and adds the new version to the File Manager contract’s list of versions using the addFileVersion function in the contract.