secondStateJS.js
which services the FairPlay - Product Giveaway site and one Javascript file ethJS.js
which services the Ethereum Search Engine Demonstration. One of the strong points of this search engine is that it allows you to create your own custom HTML/JS so that you can render your data in any way.var publicIp = "";
in the secondStateJS.js file is set to the public domain name of the server which is hosting the search engine (including the protocol) i.e.~/startup.sh
and make it executable with the chmod a+x
command. Then put the following code in the file. Please be sure to replace https://testnet-rpc.cybermiles.io:8545
with that of your RPC.crontab -e
command.harvest.py -m full
mode operates in the following way. It divides the number of blocks in the blockchain by the max_threads
setting in the config.ini file, to create chunks of blocks. It then starts a separate thread for each of those chunks. Each chunk is harvested in parallel. For example, if the blockchain has 1 million blocks and the max_threads
value is 500, there will be 500 individual threads processing 2, 000 blocks each.harvest.py -m full
mode quickly and efficiently traverses the entire blockchain with its sole purpose being to find transactions which involve smart contracts. Transactions which involve smart contract creation i.e. have a contract address in the transaction receipt are stored in the smart contract search engine's masterindex.harvest.py -m topup
mode operates in the following way. It uses the following formular to determine how many of the most recent blocks to harvest.seconds_per_block
in the config.ini is set to 10, the system will process the most recent block 1000000
and stop at block 999990
(harvest only the 10 most recent blocks). Once executed, this topup will run repeatedly i.e. it does not have to be run using cron because it already uses Python time.sleep
and will repeat as required.harvest.py -m topup
mode quickly and efficiently traverses only a few of the latest blocks with its sole purpose being to find only the most recent transactions which involve smart contracts. These are stored in the smart contract search engine's masterindex.harvest.py -m tx
mode operates in the following way. It takes all of the known ABIs which are stored in the abiindex and all of the known smart contract related transactions which are stored in the smart contract search engine's masterindex. It then creates a web3 smart contract instantiation for every combination and tests to see if web3 can sucessfully call all of the contract's public view functions.harvest.py -m faster_state
mode operates in the following way. It traverses only the most recent blocks, calls the public/view functions of the contracts in those blocks and updates the commonindex. Remembering that the commonindex is the index which provides the smart contract state data to the API.-m faster_state
mode requires that the smart contract instantiation (both the transaction hash and all associated ABIs) is already known to the smart contract search engine indices. This requires work. This mode was created for a special case whereby the search engine was required to provide real-time data for a blockchain with 1 second block intervals. Part of this special use case required that the software which was responsible for instantiating new contracts explicitly indexed the contract's ABIs and also the transaction hash of the contract instantiation. This was achieved via the submitManyAbis API call.harvest.py -m abi
mode operates in the following way. It fetches the already indexed records from the commonindex and also fetches all of the already indexed ABIs from the abiindex. It then creates web3 contract instantiations for each of the combinations. Then in relation to each combination, if the contract instance is unable to return the public view function data perfectly then the ABI and address combiination is added to the ignoreindex. This prevents the abi mode from ever checking that particular ombination out again. On the other hand, if the public view functions of the contract instance are all returned perfectly, the code will assume that this is an associated ABI i.e. a valid ABI which is part of Solidity inheritance etc. The outcome of this process will include the abiShaList being updated as well as the functionDataList seeing the addition of the new data.harvest.py -m state
mode operates in the following way. It fetches all indexed contracts from the commonindex. It reads their abiShaList and creates a web3 contract instance for each of the ABI / address combinations. Now the ABI address combinations are not questionable because they have already been through a process of making sure that they are a real relationship which can yield real data. The state mode fetches the public view data from the contract and updates the index if the data is different to what was originally stored. It also creates a local hash of the data so that when it repeats this process over and over it can compare hashes on local disk rather than remotely issuing queries to the index.harvest.py -m bytecode
mode operates in the following way. It fetches all indexed contracts and matches their bytecode (which is in theie transaction instance input) with any individual bytecode entries in the bytecodeindex​harvest.py -m indexed
mode operates in the following way. It loops through all of the indexed contracts from the commonindex and sets the indexed
value of any item in the masterindex to true
if the contract addresses from the two indices match. This is an independent script which does not have to be run, however it does help speed up performance slightly by excluding any already indexed items when the tx mode is executed./etc/apache2/apache2.conf
file and add the following./etc/apache2/sites-enabled/search-engine-le-ssl.conf
file (which was created automatically by the above "lets encrypt" command) and add the following code inside the VirtualHost
section.