How Dcrd Works And The Shortage Of Public Dcrd Nodes

Discussion in 'Technical Development' started by David, Mar 25, 2016.

  1. 2017/12/15 - Decred v1.1.2 released! → Release Notes  → Downloads
  1. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    #1 David, Mar 25, 2016
    Last edited: Mar 25, 2016
    So yesterday I delved into the workings of DCRD after learning that there are roughly about 70 public peer nodes that all of us are using to connect through and sync up to the blockchain. And I thought to myself, most people (like myself when I first started with all of this) don't even think about the work being done behind the scenes. @chappjc and I shared a similar curiosity and thus sparked an interesting conversation between the two of us - so I'd like to share that information here. This post won't be complete by any means.. it's more focused on the initialization of dcrd and the first few steps that are taken to get dcrd up and running - with a conclusion on what you can do to make your dcrd daemon help other dcrd daemons.

    My intent in writing this is to have more "public" nodes online and strengthen the core of the Decred network. If you're reading this, the odds are highly likely that you can take action to contribute!

    Let's start out by speculating on the number of active dcrd daemons. There were 3,244 airdrop participants. Let's say that half of them claimed their airdrop coins and are running dcrd nodes 24/7 to participate in stake mining, plus any new members we might have that signed up after the airdrop and are just getting started with Decred. This means we may very well have a couple thousand "leechers" and only 70 +/-10 full time seeders. Any p2p network needs a fair balance of seeders/leechers to be strong. Now I doubt these leechers are leechers by choice; it's more likely they don't know that they are not committing their nodes to support the network.

    When dcrd is first started, it's first network-related step is to contact the official DNS servers as specified in the code:
    Code:
    var mainNetParams = params{
    Params: &chaincfg.MainNetParams,
    rpcPort: "9109",
    dnsSeeds: []string{
    "mainnet-seed.decred.mindcry.org",
    "mainnet-seed.decred.netpurgatory.com",
    "mainnet.decredseed.org",
    "mainnet-seed.decred.org",
    },
    
    Note: @chappjc has a very handy page up on his domain which appears to list all of the public nodes and the software versions each node is running. I am surprised to see how many nodes running out of date software there are!

    These 4 DNS servers utilize "A" records to return public IPv4 addresses of dcrd nodes that are either: A) on a public IP by default, or B) advertising that they can be reached at a public IP (more on this below). I sent a query to each of these 4 servers and analyzed the results. At the time of writing, these servers only knew of 53 public nodes combined. This is world wide - as everyone is using the same code and therefore receiving the same DNS answers.

    So what can you do?

    Since I am intrigued by the workings of the software (and networking is my area of focus), let's look at why there are so few public nodes. The dcrd code requires your daemon to be fully aware of it's IP as it appears to the rest of the internet to participate in 2 way communication. For most users behind home routers, your daemons are clueless due to NAT. Due to the limitations and shortages of IPv4 addresses (with the small number of public nodes as evidence to this claim), most Decred participants are running dcrd with a private IP address behind a router that is performing NAT. What does this mean? Here's an example:
    If I was running a public node and the official DNS servers know about my node (which I am, and they do), it's likely that some of you guys will connect to my node to sync to the blockchain. If you connect to me but your dcrd instance doesn't know about it's public IP address, I won't tell the rest of the network about your node. You can still connect to me, but that's all that's going to happen. Your node will not help anyone else sync up to the blockchain, all because it isn't aware. I haven't seen any guides for setting up DCRD to make your node aware of it's public IP (@Shadowlance...an idea to incorporate in the docs?) other than a few brief mentions of the --externalip flag buried in various threads.

    I believe the positive impact of having hundreds of other nodes online will be huge for the Decred network. We will most likely see lower latency between dcrd nodes, faster convergence times (less forks in the chain?), distributed network load, and ease of access for new users in geographical areas where Decred hasn't yet reached. This is of course assuming that everyone keeps their software up to date, which as I have seen, is not going to happen. 17 out of 75 public nodes (22.6%) are still running v0.0.1!!

    While my explanation was lengthy, making the change is actually very simple. If your computer is connected to a typical home router, you will need to configure port forwarding, but the rest is easy.
    If you don't know whether or not you should do this, get on the machine that is running dcrd and go to: http://test-ipv6.com/
    Next, open a Command Prompt (Windows) or a Terminal (Linux/Mac).
    If you're on Windows, type:
    Code:
    ipconfig | findstr IPv4  
    If you're on Linux/Mac, type:
    Code:
    ifconfig | grep inet
    Note: If you are on Linux/Mac and you are only using wifi, replace ifconfig with iwconfig instead.

    If you don't see a result that matches the result on test-ipv6.com, your node isn't supporting the Decred p2p network. If you do see your IP, congratulations, you're either fortunate enough to have your very own public IP, or unfortunate enough to not have a router. If you do see your IP, ensure that your machine/firewall is allowing TCP port 9108 inbound, and you're done! If you don't see your real IP in the command line output, continue on.

    If you're here (and the majority of you will be), your machine is using a private IP address. You will need to do two things: configure your router to forward tcp port 9108 to your machine, and relaunch dcrd using the --externalip flag. The syntax is:
    Code:
    dcrd -u user -P password --externalip ipaddressfromtest-ipv6.com
    Here is the documentation from the binary:
    Code:
      /externalip:         Add an ip to the list of local addresses we claim to listen on to peers
    If you follow this, you will know it's working if you start seeing inbound connections on your dcrd daemon. Example:
    Code:
    [INF] BMGR: New valid peer 115.66.31.148:52400 (inbound) (/dcrwire:0.0.1/dcrd:0.0.8/)
    
    You should also see your public IP listed on chappjc's site linked above.

    As always, questions are welcome! Google is your friend for port forwarding if you don't know how, but feel free to post up if you get stuck. I am hoping to see the number of public Decred nodes increase as the Project continues to grow. Keep an eye on it with me @ https://decred.monkeyland.io/online.shtml!
     
  2. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    #2 chappjc, Mar 25, 2016
    Last edited: Mar 25, 2016
    Nice information. Thanks!

    There's a lot to comment on, but note that at least a couple of the 4 official seeds return IPv6 addresses too. I'd recommend http://test-ipv6.com for any "what is my IP" check.

    Also, the number of nodes the seeds decide to report is probably different from how many they know of.

    On the above, it would be more accurate to say "..to receive incoming connections." Since your daemon can form outgoing connections as long as your internet gateway isn't broken.

    You may post a link to our conversation if you want, assuming that's possible.
     
    David likes this.
  3. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    Thanks @chappjc. I updated the post to include test-ipv6.com.

    Since I'm not very familiar with the code, I am hoping someone who is can comment on the specifics of reporting peers. I'd imagine there is a metric calculation of some sort about who's the most reliable with the lowest latency, and then that information will be passed around. However, if nodes running v0.0.1 are still advertised through the official DNS servers, it doesn't seem that it's too picky about what it relays.

    Also, I looked for a way to post our chat, but couldn't find any way to directly link it.
     
  4. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    Since someone will inevitably suggest the --upnp option, I just want to preemptively advice against that. In fact, everyone should go to their router's settings and turn it OFF. :)
     
    Lee Sharp, drunkenmugsy and David like this.
  5. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    +1000! And if you're in there, turn off your router's WPS function too!
     
    chappjc likes this.
  6. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    #6 chappjc, Mar 25, 2016
    Last edited: Mar 25, 2016
    I actually don't recall latency playing a role in the metric's computation. I linked to the code in this thread: https://forum.decred.org/threads/dcrstats-com-decred-network-dashboard.809/page-4#post-15206
    The relevant lines showing the computation are here:
    https://github.com/decred/decred-seeder/blob/master/db.h#L37-L42

    Anyway, what I meant about reporting is just referring to how many records the DNS server component returns vs. how many "good" nodes are in the crawler's internal address database.
     
  7. sambiohazard

    sambiohazard Sr. Member

    Jan 21, 2016
    844
    372
    I wouldn't want to stake on a public fullnode. I think we should warn people not to stake on same machine on which they have a public fullnode. They can use stakepool for that but still they have to get wallet online to buy tickets. I dont have 2 machines to keep a public fullnode on one & wallet on another with a private fullnode.
    I am thinking of buying an odroid and pidrive for a public fullnode for decred & other coins but not sure if pidrive will work with odroid.
     
  8. Shadowlance

    Shadowlance Full Member

    Jan 9, 2016
    220
    155
    Male
    Thanks @David. Certainly something that should go into the docs. I'm nearly done with the RfP requirements, so I'll attach it to that. Some questions:

    How will this affect those with dynamic IPs (i.e. most people I suspect)? My IP changes constantly and as much as I'd like to run a public node, I can't always be around when it happens. I suppose you could script it, but that would involve a fair bit of work. To get people to run nodes it needs to be as simple as possible.

    What are the possible problems with staking on a public node? It's hard enough to ask people to run a single machine 24/7 for PoS (hence the coming stake pools) let alone two. I know I certainly can't do that.

    Great work from the both of you, and props to the devs for the best commented code I've ever seen :)
     
    chappjc and David like this.
  9. sambiohazard

    sambiohazard Sr. Member

    Jan 21, 2016
    844
    372
    People have told me that its a security risk to keep your coins on public node as you are accepting inbound connection from random nodes. I have assumed till now that this means a incoming connection can attack your PC and steal your coins. Is it not the case? Everyone has said that while it can be done, ideally you should refrain from that.
     
  10. v998

    v998 New Member

    Jan 28, 2016
    8
    3
    Just did some statistics work from the DNS seeds of the Decred Network.
    52 IPv4 Address are fetched from the dnsseeds.
    US consists of 28 of the 52 IPs.
    And in the 52 IPv4s, 12 of them are belong to DigitalOcean.

    I think this can represent some kind of decentralization...but not really that serious...
    ipv4.png ipv6d.png

    Looks like we lacked public nodes from Asia, Afria and Australia....
     
    chappjc and David like this.
  11. jolan

    jolan Sr. Member
    Developer

    Dec 7, 2015
    197
    226
    Male
    Decred Team Member
    This is inherited from btcd which isn't affected as much as it has a large pool of Bitcoin nodes.

    The quick fix would be to make a request to a service like ipify.org/jsonip.com and use that IP.

    Probably the proper thing to do is to use the IP that peers observe you coming from. However, you'd have to check the response from multiple peers to ensure they're telling you the truth.

    dcrd is still being synced to btcd and is a bit behind. For example dcrd lacks the peer package https://github.com/btcsuite/btcd/tree/master/peer.

    It'd probably be best to fix this in btcd and then it can be pulled in later.
     
    chappjc and v998 like this.
  12. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    Why wouldn't you want to stake while dcrd accepts inbound connections? Is it just because of what you've heard from random people on the internet? I think you are fear mongering without fully understanding the theory behind firewalls, ports, and the implementation that dcrd/dcrwallet/dcrctl uses.

    There is no room for assumptions in networking - things either are as they are - or they aren't. TCP/IP port numbers reach 65535. Several hundred of these ports are used by machines on a daily basis from well known protocols. Dozens can be used at a time for a single webpage that is loaded in your browser. Each one of these is essentially an inbound/outbound connection to/from your PC. Your coins are no more at risk when running a public node and staking than they are the moment you connect your computer to the internet. I would be more worried about physical access to my computer than a remote attack.
     
    sambiohazard and chappjc like this.
  13. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    #13 David, Mar 26, 2016
    Last edited: Mar 26, 2016
    One solution that comes to mind for users that have dynamically changing addresses is to use a VPN service. A good VPN should be 100% stable and the servers you connect to should not change their IP addresses. IMO, everyone should be using a VPN 100% of the time. When you connect to a VPN server, all your traffic is encrypted and encapsulated on your client until it reaches the destination server... in essence, the VPN server is your gateway out to the internet and that is where all your connections will "appear" to be coming from. I haven't tried this yet, but in theory it should work. Also, different VPN services may have different configurations - some may work and some may not. I will test it out using mine and report back!

    Edit: I've also come across the idea of using a dynamic DNS client to keep track of your dynamic IP. However, I don't know if the software is capable of performing DNS resolutions before connecting to peers (it doesn't have to the way it works now)...let alone even accept a URL as input for the --externalip flag. I will test this out as well and report back.

    The only downside that comes to mind is the cost of electricty. Even if your wallet is connected to an online computer and unlocked, it is not technically "on the internet". Yes it uses networking, but it's basically using a "sandbox" network on your local client. The wallet server runs on localhost which by nature is very secure. There is no way for me to steal/spoof/access your localhost interface, unless I had physical access to your computer or a remote terminal session with root access (which I haven't heard anything about since the days of WinXP, and even back then those vulnerabilities were fixed relatively quickly).

    Amazon's 1 year of free virtual computing could be a solution to the cost of electricty, but then we might not be helping Decred spread further geographically (the same can be said for using a VPN). The way I see it, having more nodes online will be a benefit no matter where they come from.

    I agree with the comments of the code. It definitely helps with understanding the way things work!
     
    BoMBeR, chappjc and zzzzzz like this.
  14. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    #14 chappjc, Mar 26, 2016
    Last edited: Mar 26, 2016
    The VPN would have to support port forwarding too (PIA does, for example). An it would be a gamble if you get the same port when you reconnect.

    Regarding dynamic DNS, externalip will resolve host names, so it is possible to use a dyndns provider. At least this is possible upstream as of 2+ years ago.

    Maybe this link only works for @David and I, but here's my original post in our Slack conversation.
     
    David likes this.
  15. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    Thank you for pointing this out. I use PIA and I would recommend them!
     
  16. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    #16 chappjc, Mar 26, 2016
    Last edited: Mar 26, 2016
    For maximum security, sure, it is advisable to close all unnecessary ports. And it is true that if you are only allowing outgoing connections from dcrd, then you don't need to open/forward 9108.

    But keep in mind that dcrd is still connecting to the Decred network (peers) without you opening up ports and setting --externalip. So, you will be communicating with whatever peers you connect to. So an attacker could still establish a connection with you if they setup their own node and waited for you to connect to them. I'm just pointing out that you don't somehow isolate yourself by disallowing incoming connections.

    At the end of day, the question is, Do you trust that dcrd does not have any security flaws? Perhaps start by ensuring that dcrd is run with a non-root account, and that the RPC servers are only listening on localhost.

    Although I assume the concern is that dcrd will somehow command your unlocked wallet to send money. I don't know if that's a plausible scenario. Or maybe an attacker could flood the listening dcrd with requests or find another way to crash it. Perhaps a dev can ease any concerns about this.
     
    sambiohazard, drunkenmugsy and David like this.
  17. drunkenmugsy

    drunkenmugsy Sr. Member
    Advocate (Reddit)

    Dec 28, 2015
    405
    218
    Male
    I think the major fear of running PoS on a public facing node is that you will be compromised in some fashion and release privileged information. Namely your wallet user/pass info. Either in CLI history or config files. Running a public node does not require a wallet with funds in it. The general reasoning is that a publicly connected computer/node that accepts inbound connections regardless of precautions taken can still be compromised. You are never 100% protected if you connect to or allow connections. Period.
    That said running PoS on a non public node does greatly reduce that possibility. You can reduce the chance of compromise in many, many ways. To many to list in one simple post. Running DCRD/dcrwallet on a firewalled non public node, meaning you dont accept inbound connections all connections must originate from your host greatly reduces the possibility of compromise. I would even venture to say up to 99.9% with that setup. Barring some type of honeypot DCRD node that you connect to that is running code to exploit DCRD/dcrwallet 99.9% of people will be safe.
     
    sambiohazard likes this.
  18. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    In what fashion would you be compromised? Unless there are any security flaws with the source code, I don't see this as a possibility. I would like to point out that dcrwallet runs on, and is controlled on, the localhost interface of the machine. Traffic on the localhost will never "leak" out and crossover into your internet interface. The only link between the localhost and the internet interface happens through the software. You'd have to go through the dcrd software (or any other software you might have running that uses the localhost interface) to access your localhost interface.

    Don't get me wrong, though. Security should be a great concern for everyone, especially when dealing with finances. But understanding the security precautions that have already been put in place (separating dcrd from the wallet utility, and using localhost which is very secure by nature) is key to having confidence in your own security.
     
    chappjc likes this.
  19. chappjc

    chappjc Full Member
    Developer Pool Operator (PoS)

    All good discussion. You have to weight the risks, but you certainly aren't unlocking your wallet to the public or anything crazy like that.

    That's exactly the attack scenario I was describing above too, and it doesn't sound far fetched to me. We discussed it here, so I assure you it has been tried and will be on Decred. But surely there are measures to sanitize and verify all messages received from peers. The network would have crumbled by this point if it were so susceptible to badly behaved nodes.
     
    David likes this.
  20. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    And this is where further inspection of the code becomes necessary. I am still confident that a honeypot node operator would not get very far in an exploit. Even with an established connection to many dcrd nodes, he'd still have to get through the dcrd software. You can't just "send a special packet" to a node and expect that node to return a wallet seed or a password or something like that. The functionality would have to exist in dcrd, and as far as I know, (please correct me if I'm wrong), dcrd doesn't handle any sensitive information. It does however have access to the RPC server on localhost port 9109 but I don't know the extent of its access without further digesting the code.
     

Share This Page