Hey avila, this is great news. We know that as the batteries get worn down, the wifi dongle and then the ethernet are some of the first things to go. If you want to untether the Pi, you might need to replace your batteries.
You can run both at the same time: you can connect the batteries and connect the power supply. Be sure to use Rasberry Pi power supply or a 2A power supply. Some folks power it off their laptops, this is a bad idea. You need the 2A power source.
Are you using the GoPiGo1 or GoPiGo2?