mako - full bitcoin implementation in C

Last update: Jan 5, 2023

Related tags

Overview

mako - full bitcoin implementation in C

Mako is a from-scratch bitcoin reimplementation, written in "almost-C89" (i.e. it can be compiled by a C89 compiler if is available -- no other C99 features are used).

Mako is more-or-less dependency-less. It only vendors lmdb, which has a remarkably small codebase. Mako aims to support any POSIX.1-2001 operating system as well as Windows XP and up.

Mako has a re-usable architecture. The core library (libmako) does no IO, and has almost every tool needed for working with bitcoin. The fullnode (libnode) is also a separate library which the final executable links to.

Development Status

Do not use mako in production. Mako is under heavy development and almost guaranteed to have a significant amount of bugs at this point in time.

The node itself is currently incomplete for various reasons, including:

A number of RPC calls are missing (notably essential things like getblocktemplate).
The entire wallet RPC is currently missing, along with the wallet itself.
Consensus & policy rules are mostly complete: mako supports softforks up to and including segwit, but not later additions like taproot.
A number of tests still need to be written.
I haven't taken the time to have mako fully sync mainnet, only signet. Mako passes all of the transaction and script test vectors from bitcoin core, but there's no telling what consensus issue may arise in its current state.

Some caveats:

Mako uses LMDB for the utxo database: on boot, it will allocate a 16G sparse file (this will also use 16G of virtual memory). If you are on a platform which does not support sparse files, 16G of actual disk space will be used.
Because the mainnet utxo database is currently several gigs (>=5G), mako will likely never be able to sync mainnet on a 32-bit machine as a 32-bit address space lacks the ability to map a 5 gigabyte file into memory.

Build & Usage (for experimentation only)

Mako currently has a very simple CMake build which doesn't properly test for features, so don't expect it to work on more obscure platforms. So far, mako has only been tested on Linux and Win32 (cross-compiled with mingw).

cmake . && make will produce two binaries: mako and makod. The arguments mimic bitcoin-cli and bitcoind respectively.

Why?

There are a few reasons mako needed to exist:

Because it is C, all data structures and primitives are written by hand and are not subject to any particular platform's implementation of them. On top of that, mako makes very sparing use of the C standard library. This makes mako more auditable than a bitcoin implementation written in C++, JS, Rust, Go, etc.
A low-level, portable, and re-usable codebase for bitcoin is useful for a number of projects.
Contrary to what some people might tell you, multiple implementations of a protocol are a good thing. In bitcoin's case, they are necessary to mitigate the harm of developer centralization.
The bitcoin protocol itself should be recorded in as many places as possible for posterity.
Mako is planned to be used as the base for a port to the handshake protocol, among other things.

Contribution and License Agreement

If you contribute code to this project, you are implicitly allowing your code to be distributed under the MIT license. You are also implicitly verifying that all code is your original work.

License

See LICENSE for more info.

Comments

Crash in OpenBSD RISC-V64 system

Running on a SiFive HiFive Unmatched board. CPU: rv64imafdc (4) @ 1.196GHz Memory: 16384MiB OS: OpenBSD 7.0 riscv64

Mako runs fine and syncs for some time but eventually core dumps:

# ./makod -prune
...
[pool] Received inv message with 21 items: blocks=0 txs=21 (52.79.99.2:8333).
[chain] Block 00000000000000001bdcb9de835d8ab5910ed6b3ddae435b99f3a98abeebee97 (303980) added to chain (txs=388 time=166.21).
[pool] Status: time=2014-06-03T08:25:17Z height=303980 progress=42.04% orphans=0 active=15420 target=0x18692842 peers=8
[pool] Received inv message with 15 items: blocks=0 txs=15 (73.168.103.27:8333).
Abort trap (core dumped)

Tell me how I can help finding out the bug. Core dump file available to send.

opened by antanst 4

Valgrind support?

I tried running in valgrind, but it failed:

...
[pool] Received 1444 headers from peer (###).
==6294== Invalid read of size 8
==6294==    at 0x122FBA: btc_pool_resolve_headers (in /mako/build/makod)
==6294==    by 0x12356E: btc_pool_on_headers (in /mako/build/makod)
==6294==    by 0x1247B1: btc_pool_on_msg (in /mako/build/makod)
==6294==    by 0x11E5F8: btc_peer_on_msg (in /mako/build/makod)
==6294==    by 0x11BE21: on_msg (in /mako/build/makod)
==6294==    by 0x11BAD2: btc_parser_parse (in /mako/build/makod)
==6294==    by 0x11BB76: btc_parser_feed (in /mako/build/makod)
==6294==    by 0x11E4AB: btc_peer_on_data (in /mako/build/makod)
==6294==    by 0x11BDD1: on_data (in /mako/build/makod)
==6294==    by 0x138944: handle_read (in /mako/build/makod)
==6294==    by 0x138EBC: btc_loop_poll (in /mako/build/makod)
==6294==    by 0x138DB3: btc_loop_start (in /mako/build/makod)
==6294==  Address 0x7e7d178 is 40 bytes inside a block of size 48 free'd
==6294==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==6294==    by 0x16D445: btc__free (in /mako/build/makod)
==6294==    by 0x11F9CA: btc_hdrnode_destroy (in /mako/build/makod)
==6294==    by 0x123043: btc_pool_shift_header (in /mako/build/makod)
==6294==    by 0x12355B: btc_pool_on_headers (in /mako/build/makod)
==6294==    by 0x1247B1: btc_pool_on_msg (in /mako/build/makod)
==6294==    by 0x11E5F8: btc_peer_on_msg (in /mako/build/makod)
==6294==    by 0x11BE21: on_msg (in /mako/build/makod)
==6294==    by 0x11BAD2: btc_parser_parse (in /mako/build/makod)
==6294==    by 0x11BB76: btc_parser_feed (in /mako/build/makod)
==6294==    by 0x11E4AB: btc_peer_on_data (in /mako/build/makod)
==6294==    by 0x11BDD1: on_data (in /mako/build/makod)
==6294==  Block was alloc'd at
==6294==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==6294==    by 0x16D3CA: btc__malloc (in /mako/build/makod)
==6294==    by 0x11F976: btc_hdrnode_create (in /mako/build/makod)
==6294==    by 0x123483: btc_pool_on_headers (in /mako/build/makod)
==6294==    by 0x1247B1: btc_pool_on_msg (in /mako/build/makod)
==6294==    by 0x11E5F8: btc_peer_on_msg (in /mako/build/makod)
==6294==    by 0x11BE21: on_msg (in /mako/build/makod)
==6294==    by 0x11BAD2: btc_parser_parse (in /mako/build/makod)
==6294==    by 0x11BB76: btc_parser_feed (in /mako/build/makod)
==6294==    by 0x11E4AB: btc_peer_on_data (in /mako/build/makod)
==6294==    by 0x11BDD1: on_data (in /mako/build/makod)
==6294==    by 0x138944: handle_read (in /mako/build/makod)
==6294== 
==6294== Invalid read of size 4
==6294==    at 0x183F88: btc_read32le (in /mako/build/makod)
==6294==    by 0x183FED: btc_murmur3_sum (in /mako/build/makod)
==6294==    by 0x14EE8D: kh_get_btc_hashset (in /mako/build/makod)
...

opened by MarcoFalke 4

Compilation Error on macos (clang?)
System: Apple M1 Pro (arm64) cmake version 3.21.4 Apple clang version 13.0.0 (clang-1300.0.29.3)

Note: This error did not appear when compiling on Arch Linux running x86-64. (gcc (GCC) 11.1.0)

Error:

[ 3%] Building C object CMakeFiles/mako_io.dir/src/io/loop.c.o ./mako/src/io/loop.c:1782:1: error: extraneous closing brace ('}')

Remedy: Deleting trailing } in loop.c:1771 resolved the error. Compilation completed successfully after this change.

However, I don't know if this is a breaking change warranting further tweaks.

Good news: I have successfully started syncing mainnet without further issue on macOS w/ Apple ARM SoC (so far).
opened by BluSyn 1
initial wallet implementation
This is basically a simplified version of the bcoin wallet. The main difference being that it is not multi-wallet, only multi-account (with special support for watch-only accounts).

It's a bit more clever about indexing tx/utxo records, which should make pagination very easy. With some minor modifications to the bitcoin core rpc api, we should have very fast querying.

Another internal difference is that querying is done strictly through iterators (iterator.c). The full set of data is never in memory during a query. This should allow the wallet to scale to potentially millions of transactions/utxos/accounts without OOMing.

RPC modifications

RPC modifications for pagination would include:

-listaccounts ( minconf includeWatchonly ) +listaccounts ( limit "after" ) -listtransactions ( "account" count from includeWatchonly) +listtransactions ( "account" limit after ) -listunspent ( minconf maxconf ["address",...] ) +listunspent ( "account" limit "after" ) -listsinceblock ( "blockhash" target-confirmations includeWatchonly) +listsinceblock ( "account" height limit )

We basically switch to a limit+after model, where the user is expected to pass the final key (whatever it may be) to the next call.

We can also make listtransactions pick ascending or descending order based on whether limit is positive or negative.

By default, none of these calls will consider watch-only outputs/txs unless a watch-only account is explicitly passed.

We'll also probably have to add a few calls to manage accounts:

+createaccount "name" ( index ) +watchaccount "name" "xpubkey" +renameaccount "oldname" "newname" +deleteaccount "name"

Other calls to consider:

+getreceiveaddress ( "account" ) +getchangeaddress ( "account" ) +listaddresses ( "account" limit "after" )

Simplified calls:

-sendtoaddress "bitcoinaddress" amount ( "comment" "comment-to" subtractfeefromamount ) +sendtoaddress "address" amount ( subfee minconf ) -sendfrom "fromaccount" "tobitcoinaddress" amount ( minconf "comment" "comment-to" ) +sendfrom "account" "address" amount ( subfee minconf ) -sendmany "fromaccount" {"address":amount,...} ( minconf "comment" ["address",...] ) +sendmany "account" {"address":amount,...} ( subfee minconf ) -getrawchangeaddress +getrawchangeaddress ( "account" ) -getbalance ( "account" minconf includeWatchonly ) +getbalance ( "account" )
opened by chjj 0

Run all tests

Run from the build directory. Your mileage may vary. E.g.:

git clone https://github.com/chjj/mako && cd mako/
mkdir build/ && cd build/
cmake ..
make all
../scripts/run_tests.sh

opened by katesalazar 0

Running mako RPC commands

Excuse my ignorance, I am trying to run commands using mako such as ./mako help or ./mako getblockhash 0 and seem to get the error "invalid arguments". I saw that some RPC commands are not implemented but I can't seem to get any working. I am making a newbie mistake?

BTW, I just compiled on MacOS Monterrey (11.0.1) and can confirm everything is working as expected. I downloaded CMake from here.

Currently syncing testnet. If you need any help testing it out on Mac feel free to reach me out through your preferred method.

Cheers, and thanks for your hard work!

opened by handsomelatino 0
docs: fix simple typo, automaticaly -> automatically

There is a small typo in src/io/http/http_parser.c.

Should read automatically rather than automaticaly.

Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

opened by timgates42 0
Bitcoin Cash community loves innovation, this project will be welcome there
I consider this project a significant and game-changing innovation, I think BCH community would love it.

While being almost a clone of BTC, we have a lot more of these kind of groundbreaking/disruptive things comparing to BTC, such as

Non-custodial escrows built into the coin (OP_CHECKDATASIG)

Schnorr signatures and fixed malleability since 2020

Advanced difficulty adjustment algorithm that actually works as designed

Multiple (5) node software implementations already present

Native introspection (coming May 2022, already confirmed)

GROUP token system (in works now, coming in 2022 probably)

And last but not least,

32MB blocks on production right now, 256MB blocks working on Testnet

Your project would we really welcome in our community, as it is very open and multiple implementations of the same network node are cherished and encouraged.

I think you would feel at home there.
opened by ShadowOfHarbringer 4
name collision

It seems there is a another project already using the name "mako".

I don't use wayland, and plan to avoid using it for as long as humanly possible so I personally don't care, but it does present a challenge to package maintainers.

Judging by their man pages, we can avoid colliding with them for the most part by renaming mako to mako-cli and moving ~/.mako to ~/.makod.

We could also rename the project itself, probably to some other FF terminology (how about atma?).

opened by chjj 7

Owner

Christopher Jeffrey (JJ)

GitHub

Bitcoin Core integration/staging tree

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.