We are happy to open up the next RFP. This is quite large and can be split/shared by multiple people. RFP0005 - Mining Protocol Development and Pool Integration Note: This RFP is open to you. You are asked to submit proposals if you are competent in this area and the money will go to you if your application is successful upon completion of the work proposed. Guidance/advice can be provided by @ceejep, @davecgh, and @jolan. Please send all proposals to: rfp2016 <at> decred.org Closing Date All proposals must be submitted by 06:00 PM UTC March 4th, 2016. Description There is not currently a standard protocol for Decred mining. Decred has many unique features not currently available in other coins, such as an extended extra nonce space and per-miner voting. The goal is to integrate Decred-specific features into an extensible protocol that allows for the easy addition of future features. The protocol should then be integrated into an open source pool and a single, open-source miner codebase that supports both OpenCL+ADL for AMD GPUs and CUDA for NVIDIA GPUs. The miner must be able to easily integrate future devices, such as FPGAs and ASICs. Requirements Familiarity with the stratum and getblocktemplate protocols, JSON data parsing, cryptocurrency miner development, OpenCL/ADL/CUDA, and proof-of-work mining pool development. Scope The first task will be to analyze existing protocols and design a protocol that is flexible enough to meet the demands of any miner. It should include the best features of both the stratum and getblocktemplate protocols. The design must be able to scale from being lightweight and low bandwidth (stratum) to full knowledge of block information (getblocktemplate). The miner should have the ability to select from the pool UI which data they will be served. As the current standard for pool/miner software is JSON, it should use this encoding standard. The protocol should make efficient use of extra nonce space and dictate which extra nonce data must be exhausted first before proceeding to the next nonce space. Extended voteBits is a 73-byte field that will eventually be held in the coinbase signature script. It will be used to provide a larger space for voting on future issues. The pool UI should give the end user the ability to store the extended voteBits for a proof of work, and all work given to them should contain their selected voteBits. When work is updated and a change to the merkle root of a block is required, the miner should be broadcast a proof of inclusion for their extended voteBits at minimum. This includes the path to the coinbase transaction, the data in the coinbase witness, and the hash of the coinbase prefix. The pool UI should allow the end user to select the verbosity of their provided work. For example, including all transaction hashes, including all transactions, including all vote transactions, and so on. The provided miner should be well commented, readable, and have the ability to easily integrate future devices. In addition, the build process must be documented in detail for the Decred Tier 1 OSes (Windows, Linux) and platforms (x86 and x86-64) and be capable of providing static binaries. Estimated Work 21-28 days FTE or greater (1 FTE = 40 hrs/week). Proposal Proposals should include: References to prior work on miners, pools, and protocols Initial overview of how you would construct the protocol Estimated completion time Payment required Milestones Completion of the initial mining protocol draft Overhaul of the draft into a final draft based on feedback from the Decred developers Integration of the protocol into a pool Integration of the protocol into an open-source miner that handles both OpenCL and CUDA implementations of BLAKE256 and supports GPU monitoring Documentation of toolchain/build environment for providing static or nearly static binaries for Linux/Windows 32-bit/64-bit Modification of these proposed milestones is acceptable, and must be accompanied by sound reasoning in a proposal. Under no circumstances shall payments be made in advance of work being completed.
90% of this RFP is already done as there is a getwork-over-tcp protocol already which works nicely with tpruvot's ccminer. This proto just needs to be imported into cg/sgminer and all the platforms etc. are covered.
I don't think a project such as this should polish the same turd that should have been retired ages ago. Anyways, while I can't and don't do the pool portion, the miner portion is something I think I could do. Something new and just for Decred, with more separation between the pool and the mining device. For GBT, I would suggest GBT over TCP rather than HTTP - HTTP was used only for backward compatibility with the horribly designed getwork protocol. While I probably wouldn't be the one to do the CUDA portions, I could likely make the base of the miner, and the OpenCL/AMD portions - as well as the interfaces for different types of miner devices. I think this should be as generic as possible, so it's simple to add support later - as well as be able to exclude/include certain backends at build time. Yeah, it would be a large project, but everyone wants to avoid doing anything more than the bare minimum because "well, cgminer is there, and it mostly works..."
I hope the work done by @tpruvot can be included into a joint proposal by @Wolf & him and they can do respectively AMD & Nvidia miners which can be then polished into a nice GUI by them or someone else. If you two are ready to work together, it will be great news for us miners.
I completely agree with Wolf here about not wanting to keep building on top of something that is really just a mess to begin with and should have been retired long ago. I personally am fully in favor of something something fresh that is designed with proper engineering principles. My personal feeling is this RFP is going to require collaboration between multiple people to be done right and as the initial statement of the RFP points out, the work can be split between multiple people. I definitely would like to see such a collaboration come to fruition.
I agree with ocminer for the moment, i/we already spent too much time on the various existing projects to restart all from scratch. Ive an upcoming sgminer which do a better hashrate than ccminer on nvidia cards, even if it was initially made for amd cards, and will be a part of my purposal. I'm not against a new project, but will take a lot of time... and money... You want a single miner... so here it is currently on (private) beta test by some different hardware owners (linux/windows/amd/nvidia)
Excellent. I hope it will also have a GUI version or someone comes in to do that part. I was waiting for your post
Doesn't support GBT, nowhere near well commented nor readable, and integrating future devices will be a nightmare and a half.
If you're going to tell me you like cgminer's code, and that it's readable, clear, and well commented, then you've been having serious hallucinations and should probably seek medical attention.
Regarding spending a lot of time on a bunch of existing implementations, that is precisely why the other currently active Decred developers and I prefer to have a single well-supported miner that allows driver modules for CUDA/OpenCL kernels, FPGA bitstreams, and custom ASICs. In my opinion, it is simply not in the project's best interest to attempt to maintain the common upper layers across multiple projects. Do keep in mind here that I'm absolutely NOT attempting to say there is no value in the community having multiple miner implementations as there absolutely is benefit in that and I'm excited to see there is activity in that regard and hope it continues. However, when speaking solely from a project perspective and what is best to maintain in the Decred repositories, a single well-maintained and well-supported miner that allows dedicated hardware vendors to write drivers for it makes the most sense. Along these lines, I was perusing the sgminer FAQ and it says: I wholeheartedly disagree with that viewpoint. The upper layers do exactly the same thing regardless of what the underlying hardware is. They need to be able to establish connections, handle TLS and proxies, parse JSON, deal with the various protocols for solo mining and pooled mining, etc. Ideally, they should also provide an RPC interface for access to those details which further enables the potential for rich analytics. It makes absolutely no sense for every GPU, FPGA and ASIC developer to implement all of those things again in their own standalone software or to have to maintain a fork which requires a huge ongoing effort to stay synced with upstream. Instead, developers for dedicated hardware products should only need to develop a specific driver to allow their device to work with the upper layers. Take a look at how Operating Systems handle devices such as the video cards themselves which perform the mining. Each device class has a unified interface for which device manufacturers create their own drivers. Every video card manufacturer most definitely doesn't have to release or fork their own version of Windows and X11. That is exactly what asking FPGA developers and ASIC manufacturers to create their own standalone mining software is doing.
look like there is no more "upstream" on sgminer... maybe the right time to fork it to a new "clminer" project
I did not have much time recently, so I was not able to finish anything from my todos (stratum mining portal,CUDA improvements, FPGA), but hopefully it will change. I'd love to help with this RFP005. Although I agree with @davecgh about "starting from scratch", I still feel that throwing away the loads of effort with present miners is a waste.
The project has received some excellent proposals for RFP-5. Due to the complexity of this RFP, there will be ongoing correspondence with everyone who submitted a proposal to find the optimal outcome that will benefit the project. Ideally, this RFP will be filled through a collaborative effort from interested parties. More details will follow here after the correspondence concludes. Stay tuned!
Great stuff _ingsoc, I truly hope that all the interested folks figure out a way to collaborate on this one, indeed a very big and complex subject overall. Good luck to all developers and applicants!
Apologies for the delay! This RFP involved a lot of discussion and planning as the scope turned out to be much larger than originally anticipated. The project would like to congratulate @Dirbaio, @ocminer, @tpruvot, and @Wolf, and thank them for their extensive proposals, amendments, and collaboration in putting together a comprehensive plan of action to meet the requirements. All of these great developers will work with C0 developers on a new mining protocol called "Haste". Specifically, @Dirbaio will: Create a Haste server as a dcrd package Integrate Haste into gominer and work with @Wolf and @tpruvot to integrate AMD, NVIDIA, and Mojo v3 hardware Integrate gominer to work with NOMP and work with @ocminer to ensure the NOMP is working properly @ocminer will: Open-source modifications to MPOS, NOMP, stratum-mining, and pushool required to operate a Decred PoW pool Integrate Haste support into NOMP so that mining pools can use it communicate with miners @tpruvot will: Open-source improvements to ccminer and the NVIDIA kernel Integrate and deslug NVIDIA GPU support for gominer @Wolf will: Open-source improvements to cgminer and the AMD kernel Fix high CPU usage when running cgminer Integrate and deslug AMD GPU and Mojo v3 support for gominer
Great news! Good luck all! BTW, I'm sure everyone involved is aware, but I just want to acknowledge that the optimizations @tpruvot made to the OpenCL Decred kernel in his sgminer fork are quite good, if somewhat not human readable . Will that be the basis for the AMD kernel? Also, it's not clear from this announcement if CUDA will be used in the implementation of the NVIDIA kernels, as in ccminer. Will gominer (what's that BTW?) be exclusively OpenCL or will it also use CUDA? (Please let CUDA be involved.)
Yes, as the RFP requires first-class support for both CUDA and OpenCL, the miner will need to work equally well on both types of hardware. In regards to gominer, it is a new miner being developed in Go which is much better suited to handle the upper level tasks that all miners, regardless of what the underlying hardware is (CPUs, AMD/nVidia GPUs, FPGAs, ASICs), have to handle such as establishing connections, handling TLS and proxies, parsing JSON, dealing with the various protocols for solo mining and pooled mining, etc. It will use interfaces both for obtaining work and for handing the work off to hardware such that the core code itself doesn't have to change when the protocols surrounding getting work and device-specific communication mechanisms are updated. The intent is to make it much easier to maintain and work with versus the current insane model of having to redo the exact same core code changes and protocol work in multiple miners because they only properly support one specific type of hardware. From a community perspective, it's good to have alternatives and I'm positive they will continue to be developed by their respective developers, however, from a project perspective, there needs to be a single miner that works equally well everywhere and provides the ability to easily add support for new hardware such as FPGAs and ASICs without asking the manufacturers to fork their own version of the miner. @Dirbaio can elaborate further since he's been working on it and will continue developing it for the RFP.