I'm setting up a repo to be used in projects where CID (related) data needs to be transacted on-chain and where I'm following a work-flow of:
1.) Establishing the CID data; 2.) Transacting said data; 3.) Publishing/Importing data in to IPFS after a successful transaction.
The main purpose of the repo is to reliably determine CIDs without importing data into IPFS(step 1). The workflow is aimed at avoiding the risk of front-running, based on data becoming publicly available before the transaction sub 2 is completed (or even initiated). My thoughts on this perceived risk, basically run down to this:
The purpose of the on-chain transaction is not only to uniquely identify content (say NFT material), but also to establish/determine its provenance, or create some form of connection with its creator/(original) owner. Cleary the public availability of such content, detracts from the core purpose of such a transaction, since it allows for - let's call it - NFT front running; one could monitor the network/specific nodes generally involved in publishing NFT(like material)to the IPFS network, for announcements on data that has been added, and claim the mentioned connection for oneself.
Question: Is this "problem" as practical as I am perceiving it? I can only find very limited information on mitigating this risk, although 1.) the practice of creating gas-less NFT's is increasingly popular and 2.) most of the data will likely enter the IPFS through pinning services (with presumably short announcement intervals) are increasingly popular (instead of via self-managed nodes where one could programatically decouple adding(pre-transaction) and pinning/announcement(post-transaction).
Issue(the main reason of this entry): I'm having difficulties establishing the CID for content exceeding the block-size.
Creating a DAG Node and adding children to it:
# parts of the test file cid-from-scratch.js
describe("Create DAG-PB root from scratch", function() {
let dagNode;
it("Creates root DAG NODE", async function() {
this.timeout(6000);
dagNode = await sliceAddLink(buffer, new DAGNode())
// Comparing the length Links property of the created root node, with that retrieved from local node
// using the same content
assert.equal(dagNode.Links.length, localDag.value.Links.length,
"Expected same amount of children in created, as in retrieved DAG ")
for (i = 0; i < dagNode.Links.length; i++) {
console.log("Children Strings: ", dagNode.Links[i].Hash.toString())
// Comparing the strings of the children
assert.equal(dagNode.Links[i].Hash.toString(), localDag.value.Links[i].Hash.toString(), "Children's CID should be same")
}
console.log(dagNode)
})
})
/**
*
* @param {Buffer} buffer2Slice The full content Buffer
* @param {Object} dagNode new DAGNode()
*/
function sliceAddLink(buffer2Slice, dagNode) {
return new Promise(async function(resolve, reject) {
try {
while (buffer2Slice.length > 0) {
let slice = buffer2Slice.slice(0, 262144);
buffer2Slice = buffer2Slice.slice(262144);
sliceAddResult = await createCid(slice, ...[, , ], 85);
let link = { Tsize: slice.length, Hash: sliceAddResult };
dagNode.addLink(link);
}
resolve(dagNode)
} catch (err) { reject(err) }
})
}
/**
*
* @param {Buffer} content
* @param {string} [hashAlg] // optional
* @param {number} [cidVersion] // optional
* @param {number} [cidCode] // optional - should be set to 85 in creating DAG-LINK
*/
async function createCid(content, hashAlg, cidVersion, cidCode) {
hashAlg = hashAlg || cidOptions.hashAlg;
cidVersion = cidVersion || cidOptions.cidVersion
cidCode = cidCode || cidOptions.code
let fileHash = await multiHashing(content, hashAlg)
return new CID(cidVersion, cidCode, fileHash)
}
The Links property of the created DAG matches that of the DAG retrieved from the local ipfs node and that from Infura (of course based on same content). The problem is that, unlike the retrieved DAG Nodes, the data field of the created DAGNode is empty(and therefore yielding a diff CID):
DAGNode retrieved: Data property containing data
DAGNode created: Data property is Empty
I'm adding to IPFS like so:
/**
* @note ipfs.add with the preset options
* @param {*} content Buffer (of File) to be published
* @param {*} ipfs The ipfs instance involved
*/
async function assetToIPFS(content, ipfs) {
let result = await ipfs.add(content, cidOptions)
return result;
}
// Using the following ADD options
const cidOptions = {
cidVersion: 1, // ipfs.add default = 0
hashAlg: 'sha2-256',
code: 112
}
it("Adding publicly yields the same CID result", async function() {
// longer then standard time out because of interaction with public IPFS gateway
this.timeout(6000);
_ipfsInfura = await ipfsInfura;
let addResult = await assetToIPFS(buffer, _ipfsInfura)
assert.equal(localAddResult.cid.toString(), addResult.cid.toString(), "Different CIDS! (expected same)")
assert.equal(localAddResult.size, addResult.size, "Expected same size!")
})
and subsequently getting the DAG (from both Local Node and Infura Node) like so(, to compare them with the created DAG):
// Differences in DAG-PB object representation in Infura and Local node!
it("dag_get local and dag_get infura yield the same Data and Links array", async function() {
this.timeout(6000);
let cid = localAddResult.cid
localDag = await dagGet(cid, _ipfsLocal);
let infuraDag = await dagGet(cid, _ipfsInfura);
console.log("Local DAG: ", localDag.value);
console.log("Infura DAG: ", infuraDag.value);
// Differences in DAG-PB object representation in Infura and Local node
// Data is Buffer in localDag and uint8array in infuraDag
assert.equalBytes(await dagData(localDag.value), await dagData(infuraDag.value), 'Expected Equal Data')
assert.equal(infuraDag.value.Links.length, localDag.value.Links.length, "Expected same amount of children")
assert(localDag.value.Links.length > 0, "Should have children (DAG-LINK objects)")
for (i = 0; i < localDag.value.Links.length; i++) {
assert.equal(infuraDag.value.Links[i].Hash.toString(), localDag.value.Links[i].Hash.toString())
}
})
IPFS Docs on the issue of work with blocks state that "A given file's 'hash' is actually the hash of the root (uppermost) node in the DAG."
You could suspect that the DAG node's Data field plays a role in this. On the other hand, looking at the length of "Data", the js-ipfs examples on dag.put ('Some data') and the IPLD DAG-PB specifications ("Data may be omitted or a byte array with a length of zero or more"), this property seems more arbitrary. (The Data array/buffer from both the infura and local ipfs node have the same content)
How can I create a root DAG node (using a content buffer and CID options), with not only the same Links property, but also the same Data property as the DAG root I'm getting after adding the same content buffer to an ipfs instance?