How segregated witness aims to fix transaction malleability problem: a talk by Julian Konchunas from Pandora Boxchain Berlin meetup https://www.meetup.com/pandoraboxchain-berlin-meetup/events/253033999/

1. Malleability and SegWit
How segregated witness aims to fix
transaction malleability problem
Julian Konchunas
Pandora Boxchain core developer
konchunas@pandoraboxchain.ai

2. Motivation
● Our Prometheus protocol is based and inspired by Bitcoin. We take chunks
of Bitcoin implementation and put them together
● As you may know Bitcoin was created 9 years ago and it has collected some
amount of technical debt because it has faced several challenges
throughout this years.
● One of such big challenges was so called transaction malleability, which
recently have been fixed by Segregated Witness. Everybody heard of it,
but not everybody knows how it works from a technical side.
● I will tell you about how it works as simple as I can, but understanding
of how payments are done in Bitcoin would be good for you
● And also I will shed some light as how to fix this problem if you are
going to create your Bitcoin-like blockchain from scratch.

3. Transactions and signing
Malleability
Witness
From scratch
Overview

4. Transactions and signing
● Simplified structure
● Transaction unlocking
● Signature forming
Malleability
Witness
From scratch

5. Simplified transaction structure
Transactions have fields
Output
Input

6. Simplified transaction structure
How much bitcoins we want to send
value
Output
Input
5

7. Simplified transaction structure
Previous transaction hash where we should take these 5 bitcoins
prev_output
value
Output
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5

8. Simplified transaction structure
Proofs that you have the rights to use funds from previous transaction
prev_output
value
Output
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
script_sig 6c9be328960c65b8dab9a50d98cda7...2aae86c937562c0c58e5962b8514301
5

9. Simplified transaction structure
prev_output
value
Output
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
OP_DUP OP_HASH160 <pubkeyHash> OP_EQUALVERIFY
Rules for unlocking this transaction bitcoins in future
script_sig 6c9be328960c65b8dab9a50d98cda7...2aae86c937562c0c58e5962b8514301
5

10. Transactions and signing
● Simplified structure
● Transaction unlocking
● Signature forming
Malleability
Witness
From scratch

11. Pay to public key hash (P2PKH)
Locks transaction by script:script_pubkey

12. Pay to public key hash (P2PKH)
OP_DUP OP_HASH160 <pubkeyHash> OP_EQUALVERIFY OP_CHECKSIG
which says “allow to use these funds to owner of specific keys”
Locks transaction by script:script_pubkey

13. Pay to public key hash (P2PKH)
script_sig
OP_DUP OP_HASH160 <pubkeyHash> OP_EQUALVERIFY OP_CHECKSIG
which says “allow to use these funds to owner of specific keys”
Locks transaction by script:script_pubkey
Unlocks transaction

14. Pay to public key hash (P2PKH)
And to prove that you have these keys you should provide:
● Signature
● Public key
script_sig
OP_DUP OP_HASH160 <pubkeyHash> OP_EQUALVERIFY OP_CHECKSIG
which says “allow to use these funds to owner of specific keys”
Locks transaction by script:script_pubkey
Unlocks transaction

15. Transactions and signing
● Simplified structure
● Transaction unlocking
● Signature forming
Malleability
Witness
From scratch

16. Take transaction fields as bytes
Transaction
prev_output
script_sig
value
script_pubkey

17. Transaction hash
Transaction
prev_output
script_sig
value
script_pubkey
tx hash
SHA-256

18. Take your private key
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256

19. Sign transaction hash
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256
sign

20. That’s a signature
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256
sign
signature

21. Derive public key
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256
sign
signature public key
derive

22. Put both in script_sig
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256
sign
signature public key
derive

23. Transactions and signing
Malleability
● Malleating
● Signature validity
● Fool the exchange
● Lightning network
Witness
From scratch

24. Malleability
● Changing of transaction hash
● Inputs and outputs stay the same
● Signature still valid
● You receive your money

25. Let’s imagine transaction
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY

26. Take its hash, sign it and push
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
PUSHDATA(72) 0450...3e01
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY
signature

27. Now push your public key
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
PUSHDATA(72) 0450...3e01
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY
PUSHDATA(33) 027c...62e7
signature public key

28. Get the transaction hash
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
PUSHDATA(72) 0450...3e01
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY
PUSHDATA(33) 027c...62e7
Transaction hash
255111d171046d4a448cafcb5886885c377afd5f87b8b015110556933cb1299a
signature public key

29. What if we put something meaningless?
Transaction hash
?????????????????????????????????????????????????????????????
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
PUSHDATA(72) 0450...3e01
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY
PUSHDATA(33) 027c...62e7OP_0 OP_DROP
signature public keymeaningless

30. Hash changed, signature still valid
Transaction hash
d3434165924ffb1182c5be4fced201dd3f7f9ddb6da775044d072d315399301c
prev_output
value
Output
script_sig
script_pubkey
Input
3a82db8f9648518144a435d5de8922ad3c4210c26cbb6cfff6a0d866323fca27
5
PUSHDATA(72) 0450...3e01
OP_DUP OP_HASH160 PUSHDATA_20 380011...53a7b4d OP_EQUALVERIFY
PUSHDATA(33) 027c...62e7OP_0 OP_DROP
signature public keymeaningless

31. Transactions and signing
Malleability
● Malleating
● Signature validity
● Fool the exchange
● Lightning network
Witness
From scratch

32. Change transaction without
changing signature
How transaction signature is still valid?

33. Change transaction without
changing signature
How transaction signature is still valid?
script_sig field is not included in hash calculation when and
verifying transaction (doing OP_CHECKSIG or OP_CHECKSIGVERIFY)

34. Change transaction without
changing signature
How transaction signature is still valid?
script_sig field is not included in hash calculation when and
verifying transaction (doing OP_CHECKSIG or OP_CHECKSIGVERIFY)
This type of malleability is called script_sig malleability

35. Let’s sign the transaction
Transaction
prev_output
empty script_sig
value
script_pubkey
Unsigned TX

36. Produce hash of unsigned TX
Transaction
prev_output
empty script_sig
value
script_pubkey
unsigned transaction
hash
SHA-256
Unsigned TX

37. Sign it with your private key
Transaction
prev_output
empty script_sig
value
script_pubkey
unsigned transaction
hash
SHA-256
signature
Unsigned TX
sign
with
private
key

38. Put signature back into script_sig
Transaction
prev_output
empty script_sig
value
script_pubkey
unsigned transaction
hash
SHA-256
signature
Transaction
prev_output
script_sig
value
script_pubkey
Unsigned TX
Signed TX
sign
with
private
key

39. Produce hash of signed TX
Transaction
prev_output
empty script_sig
value
script_pubkey
unsigned transaction
hash
SHA-256
signature
Transaction
prev_output
script_sig
value
script_pubkey
Unsigned TX
Signed TX
transaction hash
SHA-256
sign
with
private
key

40. These are different hashes!
Transaction
prev_output
empty script_sig
value
script_pubkey
unsigned transaction
hash
SHA-256
signature
Transaction
prev_output
script_sig
value
script_pubkey
Unsigned TX
Signed TX
transaction hash
SHA-256
sign
with
private
key

41. Validating transaction
When network receives new transaction to figure out if signature is
valid it takes hash of transaction without script_sig and checks
against it

42. Transactions and signing
Malleability
● Malleating
● Signature validity
● Fool the exchange
● Lightning network
Witness
From scratch

43. Back in the day...
● Go to your favourite and naive exchange
● Press withdraw money
● Remember the hash of withdrawal transaction
● Maleate transaction on your side (e.g. by mining)
● Receive your money on your wallet
● Go to exchange customer support
● Say that you cannot find transaction with such hash
● Get another withdrawal of the same amount
● ??????
● PROFIT

44. Transactions and signing
Malleability
● Malleating
● Signature validity
● Fool the exchange
● Lightning network
Witness
From scratch

45. Scalability problem
Lightning network needs guarantees that transaction hash
can not be changed

46. Lightning network
● You and your friend create a multisig transaction with 100
BTC on the balance
● Then you both create exit transactions linked to this
transaction and they stay unconfirmed
● If malleability is present your friend can change multisig
transaction before in gets confirmed
● You may lose your money not being able to issue a refund
● It makes lightning channels unreliable

47. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

48. What is this witness?
prev_output
value
script_sig
script_pubkey

49. What is this witness?
● Witness is separate transaction field
prev_output
value
script_sig
script_pubkey
witness

50. What is this witness?
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
prev_output
value
script_sig
script_pubkey
witness
kind of

51. What is this witness?
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
● But it is no longer part of transaction
identification
prev_output
value
script_sig
script_pubkey
witness
kind of

52. What is this witness?
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
● But it is no longer part of transaction
identification
● And it is no longer just script sequence
prev_output
value
script_sig
script_pubkey
witness
kind of

53. What is this witness?
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
● But it is no longer part of transaction
identification
● And it is no longer just script sequence
● It’s a stack of initial items for script
prev_output
value
script_sig
script_pubkey
witness
kind of
first second ... last

54. What is this witness?
prev_output
value
script_sig
script_pubkey
witness
kind of
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
● But it is no longer part of transaction
identification
● And it is no longer just script sequence
● It’s a stack of initial data items for script
● Last item is executable script sequence
first second ... last

55. What is this witness?
prev_output
value
script_sig
script_pubkey
witness
kind of
● Witness is separate transaction field
● You may think of it as of sophisticated
script_sig
● But it is no longer part of transaction
identification
● And it is no longer just script sequence
● It’s a stack of initial data items for script
● Last item is executable script sequence
● Script_pubkey of transaction to unlock
contains hash of last witness item
first second ... last
SHA-256

56. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

57. Good ol’ transaction hash
prev_output
value
script_sig
script_pubkey
witness
Transaction hash
aka TXID

58. Introducing brand new hash
prev_output
value
script_sig
script_pubkey
witness
Transaction hash
aka TXID
Witness transaction hash
aka WTXID

59. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

60. Take every transaction in a block
Tx witness hash Tx witness hash Tx witness hash Tx witness hash

61. Put their witness hashes in a tree
Tx witness hash Tx witness hash
Tx witness hash
Tx witness hash Tx witness hash
Tx witness hash

62. Merkelize tree to get root
Tx witness hash Tx witness hash
Tx witness hash
Tx witness hash Tx witness hash
Tx witness hash
Witness Merkle root

63. Where to put witness merkle root?
● Block header has no empty or unused fields
● But we have some field in the very first transaction
in a block
● This transaction is called “coinbase” and it is added
by miner and has mining reward
● As it creates BTC out of thin air its script_sig field
can contain 64 arbitrary bytes
● It even contains famous phrase by Satoshi in coinbase
of genesis block
● So now miners are obligated to use that space for
storing merkle root

64. Commitment structure
Tx witness hash Tx witness hash
Tx witness hash
Tx witness hash Tx witness hash
Tx witness hash
Block
Coinbase transaction
script_sig
Witness Merkle root

65. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

66. Hash public key of recipient
public key SHA-256

67. Put hash into unlocking script
OP_DUP OP_HASH160 38001...a7b4d OP_EQUALVERIFY OP_CHECKSIG
public key SHA-256

68. Hash of whole unlocking script
Hash unlocking script
OP_DUP OP_HASH160 38001...a7b4d OP_EQUALVERIFY OP_CHECKSIG
b2f34055c1f69660220426c9be328960c65b8d9a50
public key SHA-256
SHA-256

69. Prepend witness version byte
0
OP_DUP OP_HASH160 38001...a7b4d OP_EQUALVERIFY OP_CHECKSIG
b2f34055c1f69660220426c9be328960c65b8d9a50
Witness version Hash of whole unlocking script
public key SHA-256
SHA-256

70. Put this bad boy into script_pubkey
0
OP_DUP OP_HASH160 38001...a7b4d OP_EQUALVERIFY OP_CHECKSIG
b2f34055c1f69660220426c9be328960c65b8d9a50
Witness version Hash of whole unlocking script
public key SHA-256
prev_output
script_sig
value
script_pubkey
SHA-256

71. Let’s remember this tx hash
0
OP_DUP OP_HASH160 38001...a7b4d OP_EQUALVERIFY OP_CHECKSIG
b2f34055c1f69660220426c9be328960c65b8d9a50
Witness version Hash of whole unlocking script
public key SHA-256
prev_output
script_sig
value
script_pubkey
SHA-256
Transaction hash is: 182c5be4fced201dd3f7f9ddb6da775042d315399301c9ddb6d75044d072d3

72. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

73. Leave your script_sig empty
Transaction
prev_output
script_sig
value
script_pubkey

74. Hash your tx and sign it
Transaction
prev_output
script_sig
value
script_pubkey
private key
tx hash
SHA-256
sign

75. Put signature as first element of witness
Transaction
prev_output
script_sig
value
script_pubkey
Witness
private key
tx hash
SHA-256
sign
signature

76. Derive your public key
Transaction
prev_output
script_sig
value
script_pubkey
Witness
private key
tx hash
SHA-256
sign
signature
public key
derive
SHA-256

77. And put it as second witness item
Transaction
prev_output
script_sig
value
script_pubkey
Witness
private key
tx hash
SHA-256
sign
signature
public key
derive
public key
unlocking script

78. Now hash your pubkey and create
unlocking script
Transaction
prev_output
script_sig
value
script_pubkey
Witness
OP_DUP OP_HASH160 380011...53a7b4d OP_EQUALVERIFY OP_CHECKSIG
private key
tx hash
SHA-256
sign
signature
public key
derive
SHA-256
public key
unlocking script

79. Transaction
Put this script as last item
prev_output
script_sig
value
script_pubkey
Witness
OP_DUP OP_HASH160 380011...53a7b4d OP_EQUALVERIFY OP_CHECKSIG
private key
tx hash
SHA-256
sign
signature
public key
derive
SHA-256
public key
unlocking script

80. Transactions and signing
Malleability
Witness
● Transaction structure
● New type of hash
● Commitment structure
● Prepare witness locked transaction
● Pay to witness script hash
● Validation and execution
From scratch

81. Nodes validate hash of last witness
item with previous script_pubkey
OP_DUP OP_HASH160 <pubkey_hash> OP_EQUALVERIFY OP_CHECKSIG
Previous transaction
script_pubkey
SHA-256

82. When network tries to execute script
items get pushed onto stack
Script stack
signature
public key
OP_DUP OP_HASH160 <pubkey_hash> OP_EQUALVERIFY OP_CHECKSIG
Previous transaction
script_pubkey
SHA-256

83. Script stack
Last witness item gets executed
signature
public key
OP_DUP OP_HASH160 <pubkey_hash> OP_EQUALVERIFY OP_CHECKSIG
Previous transaction
script_pubkey
SHA-256

84. What do we have now?
● Transaction is unambiguously identified
○ Its unlocking script is basically hash
○ It only has values, input tx hashes and output addresses
○ It has no script commands
● Witness fields are merkelized which includes
signatures hashing for better protection
● Implementation made in a way that old clients ignore
this data so it can be deployed as soft fork
● Witness is stored in separate field in block and
so-called block weight is 4 megabytes so more
transactions can fit

85. Special note
● Explained here pay to script hash is basically pay to
public key hash wrapped into script. I used it for
clarity here because it is similar to pre-segwit pay
to public key hash which is commonly known. But
Bitcoin has special shorthand for Pay to Witness
Public key hash which saves some bytes in every
transaction and you should use it in most cases
● But this example works perfectly if you want Multisig
or just your custom script working using segregated
witness mechanism

86. Transactions and signing
Malleability
Witness
From scratch

87. How should you do it anew
● Since Witness merkle root is stored in Coinbase of
every block it may be stored as separate field
witness_merkle_root of block header
● You may skip all block_weight thing and just increase
block size to 4 megabytes which should include
transactions and witness
● If you fork you may remove every old check script
variant since code contains bunch of checks which are
not needed anymore

88. What about us?
That’s what we did for our Rustheus implementation at
Pandora Boxchain. We stripped everything which is related
to non-witness validation and you can check it out at
https://github.com/pandoraboxchain/rustheus
It’s cool, fresh and written in Rust. But keep in mind it
is work in progress!
Thank you for attention!

89. @pandoraboxchain