Use CryptoPP library for RC4 / SHA1 / MD5 algorithms instead of deploying our own implementations

[danim7]

Please tell me i'm wrong on this:

RC4Encrypt.cpp contains our own implementation of the RC4 algorithm.
SHA.cpp contains our own implementation of the SHA algorithm.
libs/common/MD5Sum.cpp contains our own implementation of the MD5 algorithm.

If we grep the code for the CryptoPP library, we can see it is only used for RSA and MD4, but not for the above algorithms:

./ClientCreditsList.cpp:#include "CryptoPP_Inc.h"	// Needed for Crypto functions
./ClientCreditsList.cpp:	delete static_cast<CryptoPP::RSASSA_PKCS1v15_SHA_Signer *>(m_pSignkey);
./ClientCreditsList.cpp:		CryptoPP::AutoSeededX917RNG<CryptoPP::DES_EDE3> rng;
./ClientCreditsList.cpp:		CryptoPP::InvertibleRSAFunction privkey;
./ClientCreditsList.cpp:		CryptoPP::FileSink *fileSink = new CryptoPP::FileSink(filename);
./ClientCreditsList.cpp:		CryptoPP::Base64Encoder *privkeysink = new CryptoPP::Base64Encoder(fileSink);
./ClientCreditsList.cpp:	} catch(const CryptoPP::Exception& e) {
./ClientCreditsList.cpp:		CryptoPP::FileSource filesource(filename2char(thePrefs::GetConfigDir() + CRYPTKEY_FILENAME), true, new CryptoPP::Base64Decoder);
./ClientCreditsList.cpp:		m_pSignkey = new CryptoPP::RSASSA_PKCS1v15_SHA_Signer(filesource);
./ClientCreditsList.cpp:		CryptoPP::RSASSA_PKCS1v15_SHA_Verifier pubkey(*static_cast<CryptoPP::RSASSA_PKCS1v15_SHA_Signer *>(m_pSignkey));
./ClientCreditsList.cpp:		CryptoPP::ArraySink asink(m_abyMyPublicKey, 80);
./ClientCreditsList.cpp:	} catch (const CryptoPP::Exception& e) {
./ClientCreditsList.cpp:		delete static_cast<CryptoPP::RSASSA_PKCS1v15_SHA_Signer *>(m_pSignkey);
./ClientCreditsList.cpp:	CryptoPP::RSASSA_PKCS1v15_SHA_Signer* signer =
./ClientCreditsList.cpp:		static_cast<CryptoPP::RSASSA_PKCS1v15_SHA_Signer *>(sigkey);
./ClientCreditsList.cpp:		signer = static_cast<CryptoPP::RSASSA_PKCS1v15_SHA_Signer *>(m_pSignkey);
./ClientCreditsList.cpp:		CryptoPP::SecByteBlock sbbSignature(signer->SignatureLength());
./ClientCreditsList.cpp:		CryptoPP::AutoSeededX917RNG<CryptoPP::DES_EDE3> rng;
./ClientCreditsList.cpp:		CryptoPP::ArraySink asink(pachOutput, nMaxSize);
./ClientCreditsList.cpp:	} catch (const CryptoPP::Exception& e) {
./ClientCreditsList.cpp:		CryptoPP::StringSource ss_Pubkey((uint8_t*)pTarget->GetSecureIdent(),pTarget->GetSecIDKeyLen(),true,0);
./ClientCreditsList.cpp:		CryptoPP::RSASSA_PKCS1v15_SHA_Verifier pubkey(ss_Pubkey);
./ClientCreditsList.cpp:	} catch (const CryptoPP::Exception& e) {
./ClientCreditsList.cpp:	CryptoPP::AutoSeededX917RNG<CryptoPP::DES_EDE3> rng;
./ClientCreditsList.cpp:	CryptoPP::RSASSA_PKCS1v15_SHA_Signer priv(rng, 384);
./ClientCreditsList.cpp:	CryptoPP::RSASSA_PKCS1v15_SHA_Verifier pub(priv);
./ClientCreditsList.cpp:	CryptoPP::ArraySink asink(abyPublicKey, 80);
./EncryptedDatagramSocket.cpp:#include "CryptoPP_Inc.h"	// Needed for Crypto functions
./EncryptedStreamSocket.cpp:		m_cryptDHA.Randomize((CryptoPP::AutoSeededRandomPool&)GetRandomPool(), DHAGREEMENT_A_BITS); // our random a
./EncryptedStreamSocket.cpp:		CryptoPP::Integer cryptDHPrime((uint8_t*)dh768_p, PRIMESIZE_BYTES);  // our fixed prime
./EncryptedStreamSocket.cpp:		CryptoPP::Integer cryptDHGexpAmodP = a_exp_b_mod_c(CryptoPP::Integer(2), m_cryptDHA, cryptDHPrime);
./EncryptedStreamSocket.cpp:					CryptoPP::Integer cryptDHAnswer((uint8_t*)aBuffer, PRIMESIZE_BYTES);
./EncryptedStreamSocket.cpp:					CryptoPP::Integer cryptDHPrime((uint8_t*)dh768_p, PRIMESIZE_BYTES);  // our fixed prime
./EncryptedStreamSocket.cpp:					CryptoPP::Integer cryptResult = a_exp_b_mod_c(cryptDHAnswer, m_cryptDHA, cryptDHPrime);
./utils/fileview/KadFiles.cpp:#include "../../CryptoPP_Inc.h"
./utils/fileview/KadFiles.cpp:	CryptoPP::Weak::MD4 md4_hasher;
./utils/scripts/sanity:	(filename =~ /CryptoPP/)
./ClientCreditsList.h:	// A void* to avoid having to include the large CryptoPP.h file
./EncryptedStreamSocket.h:#include "CryptoPP_Inc.h"	// Needed for Crypto functions
./EncryptedStreamSocket.h:	CryptoPP::Integer	m_cryptDHA;
./RandomFunctions.cpp:#include "CryptoPP_Inc.h"	// Needed for Crypto functions
./RandomFunctions.cpp:static CryptoPP::AutoSeededRandomPool cryptRandomGen;
./RandomFunctions.cpp:const CryptoPP::AutoSeededRandomPool& GetRandomPool() { return cryptRandomGen; }
./webserver/src/WebServer.cpp:#include <cryptopp/osrng.h>	// CryptoPP::AutoSeededRandomPool, for the session-token CSPRNG
./webserver/src/WebServer.cpp:		static CryptoPP::AutoSeededRandomPool s_sessionPool;
./webserver/src/WebServer.cpp:			s_sessionPool.GenerateBlock(reinterpret_cast<CryptoPP::byte *>(&fresh), sizeof(fresh));
./webserver/src/WebServer.h:	// Sourced from CryptoPP::AutoSeededRandomPool when a new session
./kademlia/kademlia/Kademlia.cpp:#include "../../CryptoPP_Inc.h"
./kademlia/kademlia/Kademlia.cpp:	CryptoPP::Weak::MD4 md4_hasher;
./KnownFile.cpp:#include "CryptoPP_Inc.h"       // Needed for MD4
./KnownFile.cpp:		CryptoPP::Weak::MD4 md4_hasher;
./RandomFunctions.h:namespace CryptoPP {
./RandomFunctions.h:const CryptoPP::AutoSeededRandomPool& GetRandomPool();

Why we are not using the CryptoPP library for ALL crypto algorithms? Is there any reason for that?
It is a bad practice in security to re-implement the crypto algorithms. And also we are surely paying a performance issue (SHA CPU instructions not being used, missing potential assembly code for certain architectures provided by the library, etc...?)

[got3nks]

You're right on both counts — re-implementing crypto when CryptoPP is already linked is a maintenance/security liability, and we are leaving SHA-NI / ARMv8 SHA crypto-ext performance on the table. ~738 LOC of hand-rolled crypto across SHA.cpp / MD5Sum.cpp / RC4Encrypt.cpp confirms it.

Background on why it's the algorithms it is (not arguing against your point, just so the protocol context is on the record): the ed2k wire format embeds MD4 (file hash in the ed2k:// URI), SHA-1 (AICH Merkle tree for block integrity), MD5 (DH→key derivation in the obfuscation layer, EC password hash, Kad node ID), and RC4 (the obfuscation cipher itself). Every other ed2k client on the network expects exactly these, so the algorithm choice is locked by protocol — but that doesn't say anything about who has to implement them. CryptoPP has them all, and your instinct is right that we should be using those.

Confirmed available in CryptoPP 8.9:

• CryptoPP::Weak::MD5 (cryptopp/md5.h, Weak1 namespace) — replaces our MD5Sum.
• CryptoPP::Weak::ARC4_Base / MARC4_Base (cryptopp/arc4.h, Weak1 namespace) — replaces our CRC4EncryptableBuffer.
• CryptoPP::SHA1 (cryptopp/sha.h, main namespace — CryptoPP didn't deprecate SHA-1 since TLS/X.509 still use it) — replaces our CSHA / AICH inner hash.

On the perf point you raised: config_asm.h defines CRYPTOPP_SHANI_AVAILABLE (Intel SHA-NI) and CRYPTOPP_ARM_SHA1_AVAILABLE (ARMv8 SHA-1 crypto extension) — both kick in transparently when the runtime CPU supports them, so the AICH hashing path would jump from software MB/s to hardware GB/s on any modern Intel/Apple Silicon CPU. That's the speedup you intuited.

Implementation cost is medium. The public interfaces of CRC4EncryptableBuffer, MD5Sum, CSHA can stay as thin wrappers that delegate internally to CryptoPP — keeps the call sites untouched and the diff bounded.

[Stoatwblr]

The mule RC4 implementation is borked and it's not worth porting to Crypto++

• It's running at CPU speed anyway
• the amount of fluffing around needed to make it work would actually result in a slower crypto++ implementation than the existing code

That said, I have a set of very rough patches which compile and run on my box quite happily. It defninitely needs an expert to massage them into a fully portable environment

[got3nks]

The mule RC4 implementation is borked and it's not worth porting to Crypto++

* It's running at CPU speed anyway

* the amount of fluffing around needed to make it work would actually result in a slower crypto++ implementation than the existing code

That said, I have a set of very rough patches which compile and run on my box quite happily. It defninitely needs an expert to massage them into a fully portable environment

I'm currently busy with the REST API (a draft is coming soon), but you can share your code and it will be helpful later. 👍

[Stoatwblr]

https://github.com/Stoatwblr/amule/blob/master/amule-SHA-MD5-patches.TGZ - The RC4 patches will bork connectivity so don't use them (they implement crypto++ but that's how I found the breakage in the obfuscation protocol)

Looking at zlib at the moment and the problem there is that zlib-ng isn't widely available, so options are limited.

The best way forward is to multithread connections, The current setup has zlib sitting singlethreaded in the main core for all connections and it's a bit of a bottleneck

[danim7]

https://github.com/Stoatwblr/amule/blob/master/amule-SHA-MD5-patches.TGZ - The RC4 patches will bork connectivity so don't use them (they implement crypto++ but that's how I found the breakage in the obfuscation protocol)

May you please check the link? I got a 404

[danim7]

I did a quick and dirty test changing from our SHA1 implementation to Cryptopplib's, and I got a nice performance improvement on a CPU with SHA instructions.

First, I created a file as:

fallocate -l 5gb test-5gb.bin

On an fresh install of ~/.aMule folder, I add the above file to the shared list.

The crypropplib's version takes 11 seconds to hash it:

2026-06-22 22:51:07: ThreadTasks.cpp(132): Hasher: Starting to create MD4 and AICH hash for file: test-5gb.bin
2026-06-22 22:51:07: SharedDirWatcher.cpp(160): KnownFileList: Shared-dir watcher enabled
2026-06-22 22:51:18: amule.cpp(1551): KnownFileList: Safe adding file to sharedlist: test-5gb.bin
2026-06-22 22:51:18: amule.cpp(1558): KnownFileList: Failsafe for crash on file hashing creation
2026-06-22 22:51:18: KnownFileList.cpp(211): KnownFileList: start saving known.met
2026-06-22 22:51:18: KnownFileList.cpp(246): KnownFileList: finished saving known.met
2026-06-22 22:51:18: KnownFileList.cpp(211): KnownFileList: start saving known.met
2026-06-22 22:51:18: KnownFileList.cpp(246): KnownFileList: finished saving known.met
2026-06-22 22:51:18: ThreadTasks.cpp(282): AICH-Hasher: Synchronization thread started.
2026-06-22 22:51:18: ThreadTasks.cpp(445): AICH-Hasher: Masterhashes of known files have been loaded.

The version with our own implementation takes 36 seconds to hash it:

2026-06-22 22:55:50: ThreadTasks.cpp(132): Hasher: Starting to create MD4 and AICH hash for file: test-5gb.bin
2026-06-22 22:56:26: amule.cpp(1551): KnownFileList: Safe adding file to sharedlist: test-5gb.bin
2026-06-22 22:56:26: amule.cpp(1558): KnownFileList: Failsafe for crash on file hashing creation
2026-06-22 22:56:26: KnownFileList.cpp(211): KnownFileList: start saving known.met
2026-06-22 22:56:26: KnownFileList.cpp(246): KnownFileList: finished saving known.met
2026-06-22 22:56:26: KnownFileList.cpp(211): KnownFileList: start saving known.met
2026-06-22 22:56:26: KnownFileList.cpp(246): KnownFileList: finished saving known.met
2026-06-22 22:56:26: ThreadTasks.cpp(282): AICH-Hasher: Synchronization thread started.
2026-06-22 22:56:26: ThreadTasks.cpp(445): AICH-Hasher: Masterhashes of known files have been loaded.

Both version generate the same known2_64 file, so the hash shall be correct.
This needs more exhaustive and formal testing and a cleaner code, but it already looks like an improvement. I don't have too much free time now (and that won't change soon...), but I will try to create a proper PR if no-one else does.

[got3nks]

Aha nice benchmark results @danim7. Already tracked in the 3.1.0 milestone. If anyone puts up a PR before I do, I'll review it.

[Stoatwblr]

https://github.com/Stoatwblr/amule/blob/master/amule-SHA-MD5-patches.TGZ - The RC4 patches will bork connectivity so don't use them (they implement crypto++ but that's how I found the breakage in the obfuscation protocol)

May you please check the link? I got a 404

03-patch-SHA.cpp

04-patch-src_libs_common_MD5Sum.cpp

These should be robustly threadsafe. I'm using them and they're stable for me