I’m making a project with VANETs (Vehicular Ad Hoc NETworks) and I’m using GoPiGo robots as my clients (mobile) while my laptop (static) will be the server. As I’m new to Python programming, I am getting a lot of help from the internet. I have written a piece of code in Python 3.6 32-bit and I’m witnessing an error when I try to send data from client to the server to be stored in a text file (.txt). Can someone please explain me where I’m going wrong. Here are the codes I’ve written:
Server code:
import socket
from threading import Thread
from socketserver import ThreadingMixIn
import os.path
TCP_IP = 'localhost'
TCP_PORT = 9890
BUFFER_SIZE = 2048
class ClientThread(Thread):
def __init__(self,ip,port,sock):
Thread.__init__(self)
self.ip = ip
self.port = port
self.sock = sock
print (" New thread started for "+ip+":"+str(port))
def run(self):
scriptpath = os.path.dirname(__file__)
filename= os.path.join(scriptpath, '/Users/Shahwani/AppData/Local/Programs/Python/Python36-32/programs/mytext.txt')
f = open(filename,'rb')
while True:
l = f.read(BUFFER_SIZE)
while (l):
self.sock.send(l)
print('Sent ',repr(l))
l = f.read(BUFFER_SIZE)
if not l:
f.close()
self.sock.close()
break
tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpsock.bind((TCP_IP, TCP_PORT))
threads = []
while True:
tcpsock.listen(5)
print ("Waiting for incoming connections...")
(conn, (ip,port)) = tcpsock.accept()
print ('Got connection from ', (ip,port))
newthread = ClientThread(ip,port,conn)
newthread.start()
threads.append(newthread)
for t in threads:
t.join()
and here comes the client code:
import socket
TCP_IP = 'localhost'
TCP_PORT = 9890
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
with open('received_file', 'wb') as f:
print ('file opened')
while True:
print('receiving data...')
data = s.recv(BUFFER_SIZE)
print('data=%s', (data))
if not data:
f.close()
print ('file close()')
break
# write data to a file
f.write(data)
print('Successfully get the file')
s.close()
print('connection closed')
Before diving into this problem, can you tell us more about the error you’re getting?
Being explicit as possible is really helpful for anyone doing debugging.
Yeah, sure. So the things is… First I open a command prompt and run server programs. It says:
Waiting for incoming connections…
Then, I run the client code in another command prompt and it says:
file opened
receiving data…
Then, the following error appears in the server cmd:
Exception in thread Thread-1:
Traceback (most recent call last):
File “C:\Users\Shahwani\AppData\Local\Programs\Python\Python36-32\lib\threading.py”, line 916, in _bootstrap_inner
self.run()
File “C:\Users\Shahwani\AppData\Local\Programs\Python\Python36-32\programs\server_multi.py”, line 22, in run
f = open(filename,‘rb’)
FileNotFoundError: [Errno 2] No such file or directory: ‘C:/Users/Shahwani/AppData/Local/Programs/Python/Python36-32/programs/mytext.txt’
First of all, I’m not sure why you’re trying to read a file that’s found in another (distant) part of the file system.
Generally, anything that has to be read within a program should be:
In the current directory of the program.
In a subdirectory of the current directory where the program is found.
And second of all, you need to check the paths to mytext.txt file. That’s why it’s failing.
Here’s an example of your 2 programs running on my Raspberry Pi just fine:
Number 1 is for the server.
Number 2 is for the client.
As the issue seems to have a solution now, do you think we can close the thread or do you have any recurring questions?
Now my program is not showing any error and both client and server files are running but whatever i type on the cmd doesn’t appear on screen and nothing’s been saved on the file! The cursor appears on the command prompt but I cannot write anything. Following is the picture of that:
Edit: Do you think it’ll be better if instead of using I/O, I directly send/receive data on the command prompt itself?
My experience with Windows' command prompt hasn’t been so satisfying.
I think one of your scripts is paused (or maybe both of them).
If I were you, I would try to hit enter in both terminals and see if the scripts resume.
You also need to make sure you have something saved in the text file that the server reads.
You can also test these scripts on your Raspberry Pi - that’s how I did it.
Do you think it’ll be better if instead of using I/O, I directly send/receive data on the command prompt itself?
I’m not sure I understand what you’re saying.
Could you rephrase the sentence?
Okay! So the current issue is: I want to use multiple clients at the same time (that means, multi-threading). So I made one server_multi.py file and two separate client_multi.py files (second being, client2_multi.py). Now, when I run the server program on command prompt and client on another command prompt, they both run and properly communicate but when I start the second client program (on yet another cmd), it’s data is not exchanged with the server, although the first client and server are still properly communicating.
I don’t know how to implement Multithreading in this scenario. Can you please guide me in this regard. Below I am attaching the server code for you guys to consider:
import socket
#import threading
def Main():
host = socket.gethostname()
port = 5200
mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print ('Socket Initialized')
mySocket.bind((host,port))
mySocket.listen(2)
conn, addr = mySocket.accept()
print ("Connection from: " + str(addr))
while True:
data = conn.recv(1024).decode()
if not data:
break
#print ("from connected user: " + str(data))
data = str(data).upper()
print ("Received from User: " + str(data))
data = input(" ? ")
conn.send(data.encode())
#conn.close()
if __name__ == '__main__':
Main()
Now, here is the client code:
import socket
def Main():
host = socket.gethostname()
port = 5200
mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mySocket.connect((host,port))
message = input(" ? ")
while message != 'q':
mySocket.send(message.encode())
data = mySocket.recv(1024).decode()
print ('Received from server: ' + data)
message = input(" ? ")
#mySocket.close()
if __name__ == '__main__':
Main()
The second client code is exactly same as the first.
You won’t be able to run multiple clients on the same server because you’re accepting a connection just once.
You’d need to run conn, addr = mySocket.accept() in a while True: loop in order to accept an “unlimited” number of clients. And yes, you’d need multiple threads.
There are 2 ways of dealing with multiple clients (at least on a small scale):
Launch a separate thread for each new client. You’d need a while loop for launching a new thread for each new connection.
By definition, this is the blocking-method way of doing it.
Go asynchronous. Use the select module for polling a list of sockets - sockets that represent the connections to the server. This method has the perk of running everything in one thread, so this means no multithreading for you @shahwani .
Here’s a nice explanation on this topic: http://steelkiwi.com/blog/working-tcp-sockets/
Here’s the documentation for the select module: https://docs.python.org/3/library/select.html
By definition, this is the non-blocking-method way of doing it.
Please let us know how far you’ve gone with your project.
I like working on servers.
I don’t know if it was correct on my part or not, to switch to another programming language when facing issues with the first one, but today I did just that.
I wrote a C++ code for multi-threaded Server and ran it in Qt 4.1.0; it’s working just fine and I am able to open three telnet clients and send data from all three of them which is properly being received at the server (an echo Server that just prints whatever it receives).
So the next target is to turn these telnet clients into a proper client code that is written in C++ and can be configured on GoPiGos. While making changes in the server code such that it not only receives whatever data is coming from the clients, but can respond as well. At the end I aim to have a bi-directional TCP socket communication between a static server and my moving clients (GoPiGos).
I would ask you why have you gone on the C++ route?
As far as it goes, with Python you can develop a lot faster than with C++ and that is really good for prototyping things. C++ is used when you need every bit of performance or if you do heavy math things.
I’m curious to see how your final solution is going to look like.
It’s great to read your response. Yes, it does sound a bit odd that one would pick C++ instead of Python for such a sophisticated task. But what I was thinking is that: I want to become a good enough programmer. Since I’m quite a beginner and I am not very good at the thinking level, I believe starting with a low-level language like C/C++, would help me have stronger foundations. Once I start thinking like a programmer and I am able to build concepts at the basic level, it’ll be easier to switch to Java or Python or similar higher level languages. This is a project which I’m doing for my Master’s degree, so by the time I’m done with the project, I want to reach a certain level at such specific Programming. I hope you’re getting my gist.
Can someone please refer me some guidelines/ sources that’ll help me write code for Multi-threaded TCP Client/Server bi-directional communication (including I/O operations to store data on disk) in C++ ?
I totally get you.
That’s how I’ve started too. And it’s really good to be motivated to do things from top to bottom.
I’m curious, what kind of project are you doing? Is it a messenger application?
As for resources / guidelines on how to write code for this stuff, I’d suggest you the following:
Absolutely! As I’m reading more and more, I am realising it’s too much to learn but I don’t have enough time.
So now, what I am thinking is restricting myself within the context of my Project so that I just study what is absolutely required to fulfil the task. Otherwise, I won’t be able to complete it within stipulated time.
MY PROJECT: I have to build an emergency response system, based on Vehicular ad-hoc network. It means: There is a static server (my laptop) which is communicating with multiple moving clients (GoPiGo) and the communication between them is bi-directional. I’ll also be making a User Interface (GUI) which will display all the messages from client(s) and their response from the server. Also at the back end, I will maintain storage of those messages (for later use) along with a list of all active clients at any given time.
I shall interface a hardware pad on each GoPiGo robot; so that whenever a ‘1’ is pressed, for instance, it will trigger a certain message to the server, ‘2’ will send some other message/alert, so on and so forth.
These clients will also exchange their physical coordinates with the server (which means I have to add a GPS module and/or code somehow) by which Server will know of their exact positions. I also have to consider how I shall perform routing of the packets (I am planning to use 802.11ac for that.) I shall connect WiFi dongles on the USB port of the GoPiGo bots for the testing purpose.
Now that I have given a brief explanation of what I intend to achieve with this project; do you have any questions, feedback, suggestions (both in terms of programming or hardware) ??? Kindly let me know, may be something that shall help me achieve this task in a better way and ON TIME!
Are you going to store all the records in a database?
That should be a really useful thing to do.
Generally, I prefer offering big-picture suggestions as opposed to technical views.
I would try to concentrate on not pre-optimizing the code / architecture. That’s the most common thing I’ve seen: getting lost in the details without getting the project finished. Don’t get me wrong, I’m not suggesting you build low-quality stuff. Quite the opposite: go for the highest quality and build something you can get proud with.
I’m advising you to accept the fact that you need multiple iterations in order to get that polished code / project. Getting it right the first time is not possible, so by taking this route, you’ll save lots of time.
Also, for every small change you bring to your project, test EVERYTHING - even if it takes you hours. I’d suggest you use automated tests.
Always think BIG. Think as if you develop a product that’s going to be needed for years to come. It needs to be reliable, scalable and simple to use. I think this comes with attitude, so you may or may not have this mindset.
If you ever see a bug and you know how to fix it fast, DON’T DO IT. Always check if it fits your architecture and if nothing gets “violated”. It’s going to help you a lot in the long run - you’re going to get the habit of writing clean, architecture-compliant code.
I think this is all I can think of for now.
I told you all these suggestions from my experience, so they are already “checked”.
If there’s anything else I can help you @shahwani, please tell me - I’d be really happy to talk more with you about this project.
I have written a Multi-threaded server using pre-defined QTcpServer and QTcpSocket classes (can’t attach the files ). I also have written a code to read the file (.txt) from the server and write that on another file. Now I have to merge the two together, that is, whenever a client requests a connection; the server checks the list of recognized/authenticated clients and if the IP matches; it allows the connection and store the information in a new file, of Active clients.
P. S. Someone suggested me to use JSON for this purpose but I do not know anything about that!
For server/client stuff I would have gone with NodeJS.
It’s asynchronous in its nature and is mainly used for creating servers - it’s really simple to do that.
As for identifying your clients, I would use the server to assign unique identifiers for each client.
Each client would then be responsible to keep that user ID locally so they can communicate with the server later on, without the necessity of assigning a new ID.
Using IPs for identifying clients is generally a bad idea since IPv4 is still widely used and as we already know, IPs“repeat” themselves - I’m talking about the private IPs, such as the ones in the 192.168.0.0-192.168.255.255 range.
See RFC1918 for more information on private networks.
Now, instead of using a (.txt) file from the hard disk to store the IP’s of recognised clients, I am trying to use QList but it’s not giving me desired output.
server.cpp:
#include "myserver.h"
#include "ioprogram.h"
#include <QTcpSocket>
#include <string>
#include <iostream>
using namespace std;
MyServer::MyServer(QObject *parent): QTcpServer(parent)
{
QStringList accepted_ip_list; //List of remote IPs that can be accepted in QString list format
accepted_ip_list.append("127.0.0.1"); // IPv4 local address
accepted_ip_list.append("::1"); // IPv6 local address
accepted_ip_list.append("172.18.48.139");
// Convert from QString to integer format, generating new list
foreach (const QString &ip, accepted_ip_list)
{
QHostAddress host_address(ip);
my_accepted_ip_list.append(host_address);
}
/*
myserver = new QTcpServer(this);
connect(myserver, &QTcpServer::incomingConnection, this, &MyServer::incomingConnection);
myserver->listen(QHostAddress::Any, 1234);
*/
}
void MyServer::startServer()
{
if(!this->listen(QHostAddress::Any,1234))
{
qDebug() << "Could not start server.";
}
else
{
qDebug() << "Listening...";
}
}
void MyServer::incomingConnection(qintptr socketDescriptor)
{
QTcpSocket *socket = new QTcpSocket(this);
socket->setSocketDescriptor(socketDescriptor);
qDebug() << socketDescriptor << "Connecting...";
QHostAddress host_address = socket->peerAddress();
//quint32 ipv4 = host_address.toIPv4Address();
std::string ipv4 = host_address.toString().toStdString();
QByteArray ipv6 = QByteArray((char*)host_address.toIPv6Address().c, 16);
IOProgram test;
bool status= test.checkAuthenticity(ipv4);
if(status == false){
/*
while (myserver->hasPendingConnections())
{
QTcpSocket *socket = myserver->nextPendingConnection();
QHostAddress host_address = socket->peerAddress();
bool contains = false;
*/
for(int i=0; i < my_accepted_ip_list.size(); i++)
{
//quint32 accepted_ipv4 = my_accepted_ip_list[i].toIPv4Address();
std::string accepted_ipv4 = my_accepted_ip_list[i].toString().toStdString();
if(accepted_ipv4 == ipv4)
{
status = true;
qDebug() << "testing...";
break;
}
/* else
{
qDebug() << "Rejected!";
socket->abort(); // Reject peer by disconnecting it
socket->deleteLater(); // Schedule the socket removal from memory
}
*/
/*if(my_accepted_ip_list[i].isEqual(host_address,QHostAddress::ConvertV4MappedToIPv4))
{
contains = true;
break;
}*/
}
if(!socket->QTcpSocket::ConnectedState){
socket->abort(); // Reject peer by disconnecting it
socket->deleteLater(); // Schedule the socket removal from memory
}
if(status)
{
qDebug() << "Accepted!";
MyThread *thread = new MyThread(socketDescriptor, this);
connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
thread->start();
}
}
}
and here is my ioprogram.cpp file:
#include "ioprogram.h"
#include <QFile>
#include <QDebug>
#include <QTextStream>
#include <iostream>
#include <fstream>
#include <stdlib.h>
#include <string>
#include <vector>
#include <sstream>
using namespace std;
struct Data {
string Name;
string IP;
string Port;
};
vector<Data> AllData;
IOProgram::IOProgram()
{
}
void Read()
{
QFile file("C:/Users/Shahwani/Documents/Bremen/Sem3/Qt/MultiServerAugust/NodesList.txt");
if(file.open(QIODevice::ReadWrite | QIODevice::Text))
{
QTextStream stream(&file);
QString line;
do
{
line = stream.readLine();
qDebug() << line;
}while(!line.isNull());
file.close();
}
}
string IOProgram::inpop()
{
ifstream inFile;
inFile.open("NodesList.txt");
// inFile.clear();
// Check for error
if(inFile.fail()){
cerr << "Error opening file" << endl;
exit(1);
}
int count = 0;
string line;
char delim = '\t';
getline (inFile,line);
// Read the file until the end
while(!inFile.eof())
{
getline (inFile,line);
string token;
stringstream stream(line); // Splitting a string using a delimiter
vector<string> vect;
while(getline(stream, token, delim) )
{
vect.push_back(token);
}
Data d;
d.Name = vect[0];
d.IP = vect[1];
d.Port = vect[2];
vect.clear();
AllData.push_back(d);
count++;
}
cout << count << " nodes found!" << endl;
inFile.close();
// Check for authenticity and only allow the recognised IPs
// Write the active nodes in a separate file on the disk
ofstream outFile;
outFile.open("sample.txt");
Data activeList;
for(int i = 0; i < AllData.size(); i++)
{
activeList = AllData[i];
outFile << "Name: " << activeList.Name << "\tIP: " << activeList.IP << "\tPort: " << activeList.Port <<endl;
}
outFile.close();
return activeList.IP;
}
bool IOProgram::checkAuthenticity(string ip)
{
for(int i=0;i<AllData.size();i++)
{
Data check;
check = AllData[i];
if(ip == check.IP)
{
return true;
}
else
{
return false;
}
}
}
There is a multi threaded TCP server that is constantly listening for client connection requests. Whenever there is a remote client that requests to connect, the server should create a socket connection, the client sends the first message after connection that contains its Name, IP and Port Number. If these details match with the pre-stored registered clients at the server end, the server allows further communication; otherwise, terminates the connection. I want the server to have an ease, somehow, of appending any new IP in ‘registered clients list’ whenever is required, without a lot of trouble in modifying the code.
I hope you’re getting what I’m trying to achieve here.
.
). I also have written a code to read the file (.txt) from the server and write that on another file. Now I have to merge the two together, that is, whenever a client requests a connection; the server checks the list of recognized/authenticated clients and if the IP matches; it allows the connection and store the information in a new file, of Active clients.