29.11.2014 Views

Socket Programming - EE&T Lecture Notes

Socket Programming - EE&T Lecture Notes

Socket Programming - EE&T Lecture Notes

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

The <strong>Socket</strong>s API<br />

TELE3118 lecture notes<br />

© by Tim Moors, April 6, 2011<br />

Copyright © 06/04/11, Tim Moors


2<br />

Resources<br />

2 books:<br />

• Short & sweet (147 pages)<br />

o<br />

o<br />

o<br />

M. Donahoo and K. Calvert:<br />

TCP/IP <strong>Socket</strong>s in C: Practical Guide for Programmers<br />

, Elsevier, 2001<br />

Click on title for free online access from within UNSW<br />

Code etc: http://cs.baylor.edu/~donahoo/#series<br />

• Definitive & exhaustive (991 pages)<br />

o<br />

W. Stevens et al: Unix Network <strong>Programming</strong>, Vol.<br />

1: The <strong>Socket</strong>s Networking API, 3 rd ed., Addison-<br />

Wesley 2004<br />

Copyright © 06/04/11, Tim Moors


3<br />

Transport layer (in 1 crowded slide)<br />

Many apps need more than network layer<br />

packet delivery service.<br />

Transport protocols fix the mismatch, e.g.<br />

port #s to distinguish multiple processes<br />

• Services need ‘well known’ port #s<br />

(e.g. http=80, dns=53, ntp=123)<br />

• Clients can let operating system choose<br />

arbitrary “ephemeral” port # = return address,<br />

e.g. which web browser window<br />

Reliability<br />

• check integrity of received info<br />

(What if router mangles payload held in its network layer buffer?)<br />

o One transport protocol, UDP, only does this.<br />

• recover from loss<br />

o<br />

o<br />

o<br />

Another transport protocol, TCP, also does this.<br />

TCP’s mechanism involves 2 ends agreeing on transfer state<br />

• TCP is “connection-oriented” (UDP “connectionless”)<br />

• TCP can only unicast (UDP can multicast/broadcast)<br />

Flow control avoids loss due to excess speed,<br />

but may involve sending smaller data units<br />

=> TCP may alter message boundaries<br />

(e.g. app writes “TELE3118”, TCP might send “TELE” “31” “18”)<br />

=> TCP STREAM service vs UDP DGRAM service<br />

Copyright © 06/04/11, Tim Moors<br />

web app<br />

http dns<br />

tcp<br />

ip<br />

udp<br />

clock<br />

ntp<br />

port<br />

protocol


4<br />

<strong>Socket</strong>s history<br />

Operating System (OS) should abstract<br />

away details of hardware devices<br />

e.g. Unix provides consistent file interface<br />

for disk & keyboard & screen I/O<br />

1983: 4.2BSD Unix extended I/O to<br />

support network access via “sockets”<br />

1983: Before IP won over OSI, SNA etc<br />

=> “multiprotocol” (but small addresses)<br />

1983+Unix: C interface, e.g. pass<br />

pointers & typecast rather than<br />

overload functions.<br />

(e.g. (sockaddr*)sock_in)<br />

user<br />

space<br />

OS<br />

hardware<br />

web app clock<br />

http dns ntp<br />

sockets<br />

tcp udp<br />

ip<br />

1992: Microsoft Windows <strong>Socket</strong>s (“Winsock”) v1.0 simplifies<br />

porting network apps to Windows, with slight differences since<br />

Windows ≠ Unix.<br />

1999: IPv6 requires sockets extensions [RFC 3493]<br />

Bottom line: We want to write portable modern apps, not to read<br />

all apps using varied functions from past 25 years.<br />

For simplicity, we’ll omit error checking!<br />

The sockets API faces challenges from modern systems, as described in<br />

G. Neville-Neil: “Whither <strong>Socket</strong>s?”, ACM Queue Magazine, 7(4):34-5, May 2009<br />

Copyright © 06/04/11, Tim Moors


5<br />

Web client<br />

// Web client. Copyright (C) 2009, Tim Moors.<br />

// Compile with tim_sockets.cpp & link in Ws2_32.lib (Windows), libnsl (Unix)<br />

#include "tim_sockets.h"<br />

#include <br />

#ifdef _WIN32<br />

# define snprintf _snprintf<br />

#endif<br />

int main(int argc, char **argv) {<br />

if (argc != 3) {<br />

fprintf(stderr, "Usage: %s server-addr filename", argv[0]);<br />

exit(EXIT_FAILURE);<br />

}<br />

SOCKET s = socket(PF_INET, SOCK_STREAM, 0);<br />

struct sockaddr_in server;<br />

server.sin_family = AF_INET;<br />

server.sin_addr.s_addr = inet_addr(argv[1]);<br />

server.sin_port = htons(80);<br />

if(connect(s, (struct sockaddr *)&server, sizeof(server))==0) {<br />

#define MAX_REQ_LEN 1000<br />

char request[MAX_REQ_LEN];<br />

snprintf(request, MAX_REQ_LEN, "GET %s HTTP/1.1\nHost: %s\n\n", argv[2], argv[1]);<br />

send(s, request, strlen(request), 0);<br />

// receive file and write to stdout<br />

while (1) {<br />

#define BUF_SIZE 1234<br />

char buf[BUF_SIZE];<br />

int bytes = recv(s, buf, BUF_SIZE, 0);<br />

if (bytes


6<br />

Web server<br />

// Web server. Copyright (C) 2009, Tim Moors.<br />

// Compile with tim_sockets.cpp & link in Ws2_32.lib (Windows), libnsl (Unix)<br />

#include "tim_sockets.h"<br />

#include <br />

#define BUF_SIZE 3118<br />

int main(int argc, char *argv[])<br />

{<br />

SOCKET s = socket(PF_INET, SOCK_STREAM, 0);<br />

struct sockaddr_in endpoint;<br />

endpoint.sin_family = AF_INET;<br />

endpoint.sin_addr.s_addr = htonl(INADDR_ANY);<br />

endpoint.sin_port = htons(80);<br />

bind(s, (struct sockaddr*)&endpoint, sizeof(endpoint));<br />

listen(s, 5);<br />

while(1) {<br />

FILE* f;<br />

SOCKET new_client = accept(s, 0, 0);<br />

char buf[BUF_SIZE];<br />

recv(new_client, buf, BUF_SIZE, 0);<br />

char* method = strtok(buf, " "); // method<br />

char* status;<br />

char OK[] = "HTTP/1.1 200 OK\n\n";<br />

if(strncmp(method, "GET", 3)!=0)<br />

status = "HTTP/1.1 405 Method not allowed";<br />

else {<br />

char* uri = strtok(NULL, " ");<br />

f = fopen(uri, "r");<br />

if(f==NULL)<br />

status = "HTTP/1.1 404 Not found\n\n";<br />

else<br />

status = OK;<br />

}<br />

send(new_client, status, strlen(status), 0);<br />

if(status == OK) {<br />

while(!feof(f)) {<br />

int len = fread(buf, sizeof(char), BUF_SIZE, f);<br />

send(new_client, buf, len, 0);<br />

}<br />

fclose(f);<br />

}<br />

close(new_client);<br />

}<br />

}<br />

int main(int argc, char *argv[])<br />

{<br />

SOCKET s = socket(PF_INET, SOCK_STREA<br />

bind(s, (struct sockaddr*)&endpoint, sizeof<br />

listen(s, 5);<br />

while(1) {<br />

SOCKET new_client = accept(s, 0, 0);<br />

recv(new_client, buf, BUF_SIZE, 0);<br />

f = fopen(uri, "r");<br />

send(new_client, status, strlen(status), 0)<br />

if(status == OK) {<br />

while(!feof(f)) {<br />

int len = fread(buf, sizeof(char), BUF_S<br />

send(new_client, buf, len, 0);<br />

}<br />

}<br />

close(new_client);<br />

}<br />

}<br />

Copyright © 06/04/11, Tim Moors


7<br />

Building sockets applications<br />

Headers:<br />

• Windows: <br />

for *info() and types<br />

• Unix: <br />

<br />

<br />

for *info() and types<br />

Functions/types vary with operating system.<br />

Masked here in purple.<br />

Here Linux Windows<br />

SOCKET int SOCKET<br />

close close closesocket<br />

ioctl ioctl ioctlsocket<br />

inet_ntop inet_ntop InetNtop (Vista or later)<br />

SHUT_RD SHUT_RD SD_RECEIVE<br />

SHUT_WR SHUT_WR SD_SEND<br />

SHUT_RDWR SHUT_RDWR SD_BOTH<br />

Link in:<br />

• Windows: Ws2_32.lib<br />

& remember to call WSAStartup()/WSACleanup()<br />

• Unix: libnsl, e.g. “gcc –lnsl foo.c”<br />

Copyright © 06/04/11, Tim Moors


8<br />

Outline: 8 essentials<br />

Creation:<br />

socket(), setsockopt()<br />

Communication: send(), recv()<br />

sendto(), recvfrom()<br />

Addressing:<br />

sockaddr, sockaddr_storage<br />

Specifying addrs: bind(), connect()<br />

Serving clients:<br />

Converting #s:<br />

Names↔Addr:<br />

Cleaning up:<br />

listen(), accept()<br />

htonl(), ntohl()<br />

htons(), ntohs()<br />

getaddrinfo()<br />

shutdown(), close()<br />

Copyright © 06/04/11, Tim Moors


Copyright © 06/04/11, Tim Moors<br />

9


10<br />

socket()<br />

Provides a context (protocol & other info bound to socket)<br />

SOCKET socket(int socket_family, e.g. PF_INET, AF_INET6<br />

int socket_type, e.g. SOCK_STREAM or SOCK_DGRAM<br />

int protocol 0 lets OS choose; or IPPROTO_TCP etc<br />

);<br />

Returns: SOCKET that can be used in later I/O to identify which<br />

socket is being accessed<br />

get/setsockopt() controls socket options<br />

int setsockopt(SOCKET s,<br />

int level, e.g. SOL_SOCKET or IP_PROTOTCP<br />

int optname, e.g. SO_REUSEADDR, IP_ADD_MEMBERSHIP<br />

const void *optval, socklen_t optlen);<br />

Copyright © 06/04/11, Tim Moors


11<br />

Communicating through sockets<br />

Linux apps can apply standard file I/O to sockets:<br />

ssize_t write(int fd, const void *buf, size_t count);<br />

ssize_t read(int fd, void *buf, size_t count);<br />

But Windows doesn’t support read()/write()<br />

By default, sockets I/O functions block: Don’t return until action done.<br />

• write/send: buf passed to transport layer<br />

• read/recv: new data available<br />

Specialist socket functions give flags control & Windows compat.:<br />

ssize_t send(SOCKET s,const void *buf,size_t len,int<br />

flags);<br />

ssize_t recv(SOCKET s, void *buf, size_t len, int<br />

flags);<br />

e.g. MSG_DONTWAIT flag to prevent this function call from blocking<br />

But with which endpoint (address+port) are we talking to/from?<br />

• SOCK_DGRAM can specify/ID endpoints for individual datagrams:<br />

ssize_t sendto(SOCKET s, const void *buf, size_t len, int<br />

flags, const struct sockaddr *to, socklen_t tolen);<br />

ssize_t recvfrom(SOCKET s, void *buf, size_t len, int<br />

flags, struct sockaddr *from, socklen_t *fromlen);<br />

• SOCK_STREAM: bind() can specify local end &<br />

connect() specify server end. (more to come)<br />

Copyright © 06/04/11, Tim Moors


Identifying endpoints with sockaddr<br />

12<br />

Must identify<br />

“local” end (on computer running this process) and<br />

“remote” end (on other computer).<br />

Either may be client/server or source/destination.<br />

Do so with sockaddr struct. Aims to be protocol independent=><br />

1 st member IDs family & rest IDs endpoint.<br />

Functions defined in generic terms => typecast: (sockaddr*)&sin<br />

struct sockaddr {<br />

unsigned short sa_family;<br />

char sa_data[14];<br />

};<br />

in_addr gives access to IPv4<br />

address, most usefully as a<br />

32b word .sin_addr.s_addr<br />

14B data is too small for IPv6=>generalise to sockaddr_storage<br />

large enough to hold any address that the host supports.<br />

struct sockaddr_storage {<br />

struct sockaddr_in6 {<br />

short ss_family;<br />

short sin6_family;<br />

char __ss_pad1[_SS_PAD1SIZE]; u_short sin6_port;<br />

__int64 __ss_align;<br />

u_long sin6_flowinfo;<br />

char __ss_pad2[_SS_PAD2SIZE]; struct in6_addr sin6_addr;<br />

};<br />

u_long sin6_scope_id; };<br />

Copyright © 06/04/11, Tim Moors<br />

struct sockaddr_in{<br />

short sin_family;<br />

unsigned short sin_port;<br />

struct in_addr sin_addr;<br />

char sin_zero[8];<br />

};


13<br />

bind()<br />

int bind(SOCKET s, const struct sockaddr<br />

*my_addr, socklen_t addrlen);<br />

• Specifies the local sockaddr<br />

o Port: When app (e.g. client) doesn’t care about<br />

o<br />

value, 0 asks OS to choose an ephemeral port<br />

IP address: App that doesn’t know local address or<br />

doesn’t care which interface can specify INADDR_ANY<br />

(IN6ADDR_ANY_INIT for IPv6)<br />

• Interface may matter for multihomed machines (e.g.<br />

WiFi+Eth; firewall)<br />

• bind() fails if sockaddr is in use.<br />

o<br />

o<br />

e.g. bind() may fail when restarting program just<br />

after it closed since TCP conn. is in TIME_WAIT state.<br />

Solve with SO_REUSEADDR socket option.<br />

Copyright © 06/04/11, Tim Moors


14<br />

connect()<br />

Specifies remote end. Informs OS of destination address<br />

for sending, and how to filter/route incoming packets.<br />

int connect(SOCKET s, const struct sockaddr<br />

*serv_addr, socklen_t addrlen);<br />

Typically used with SOCK_STREAM<br />

• Starts TCP handshake with remote end (in addition to<br />

informing local OS). Returns when handshake is<br />

finished (e.g. may be ECONNREFUSED or ENETUNREACH)<br />

• If client didn’t set local endpoint (e.g. with bind()) then<br />

OS will set it like INADDR_ANY & use ephemeral port<br />

Sometimes used with SOCK_DGRAM:<br />

• Specifies default destination for send() &<br />

• Restricts recv() to remote.<br />

• Can change remote end by calling connect() again.<br />

• Yes, connectionless UDP can connect()!: UDP is<br />

connectionless at the transport layer, but connect() can be<br />

used to set state info for the sockets API<br />

Copyright © 06/04/11, Tim Moors


15<br />

listen()ing for connections<br />

By default, sockets support clients that initiate, rather<br />

than respond to, connection requests<br />

TCP: If receive SYN (= start handshake request) then send RST<br />

listen() tells OS that app is a server & specifies backlog<br />

TCP: If receive SYN then send SYN+ACK<br />

int listen(SOCKET s, int backlog);<br />

• Servers usually call bind() before listen() to specify<br />

well known port.<br />

• OS queues clients when multiple try to connect<br />

=> “backlog” = max # of ”pending connections” =<br />

handshake in progress or complete.<br />

Backlog numerology: Often 5 (max for 4.2BSD). OS may<br />

add a fudge factor (Linux: +3) to include handshakes in<br />

progress when app treats backlog as handshakes<br />

complete. Hard to set properly [Stevens UNP pp. 106-7]<br />

• listen() returns promptly, when OS is ready;<br />

accept() indicates when clients are ready<br />

Copyright © 06/04/11, Tim Moors


16<br />

accept()ing connections<br />

accept(): Respond to handshake request<br />

from client.<br />

To concurrently serve multiple clients,<br />

must distinguish communication with<br />

each => create a new socket for each.<br />

SOCKET accept(SOCKET s, struct<br />

sockaddr *addr, socklen_t<br />

*addrlen);<br />

o addr identifies client<br />

o Return value = new socket<br />

Copyright © 06/04/11, Tim Moors


17<br />

Number conversion<br />

• In Jonathan Swift’s “Gulliver’s Travels”,<br />

Lilliputians and Blefuscudians fight a<br />

war over the (trivial) issue of which end<br />

to break their eggs at (big or small)<br />

• Computing: When storing integers,<br />

which byte to store in lower address?<br />

A1: Intel et al: Little Endian (least significant)<br />

A2: IBM et al: Big Endian<br />

• Networking: Send big or little end of integer first?<br />

A: Must standardise: Big Endian.<br />

• <strong>Socket</strong>s uses Big Endian network format, which may differ<br />

from host format.<br />

=> functions to convert hton or ntoh for short (port) or<br />

long (32b IPv4 address) integers<br />

htonl(), ntohl(), htons(), ntohs()<br />

• Be careful: Compiler doesn’t usually type check, e.g.<br />

sa.sin_port=0xCAFE = 0xCAFE or 0xFECA depending on<br />

machine. Use sa.sin_port=htons(0xCAFE)<br />

Image from www.laputan.org/<br />

images/pictures/little-endian.jpg<br />

Similarly, be careful with structs, since different hosts/compilers may align members<br />

differently. Write functions to send/receive all member values, rather than directly transferring<br />

memory occupied by struct. Copyright © 06/04/11, Tim Moors


18<br />

Presenting addr’s to humans (1)<br />

• sockaddr stores addresses in numeric (binary) form.<br />

Want to present sin*_addr members in humanreadable<br />

form, e.g. “dotted decimal” for IPv4.<br />

(sin_port #s are readily presented as integers.)<br />

inet_[np]to[pn](): (n = numeric, p = presentable/printable)<br />

Protocol-independent & available under Linux and<br />

Windows Vista<br />

const char *inet_ntop(int af, const void *src,<br />

char *dst, socklen_t cnt);<br />

o src points to in_addr or in6_addr<br />

o Return value (points to dst if successful) useful when<br />

including as an argument, e.g. in printf<br />

o Capitalised as “InetNtop()” under Windows Vista<br />

int inet_pton(int af, const char *src, void<br />

*dst);<br />

Windows also has its own functions, e.g. WSAStringToAddress() is analogous to inet_pton()<br />

Copyright © 06/04/11, Tim Moors


19<br />

Presenting addr’s to humans (2)<br />

inet_[na]to[an](): Earlier IPv4 functions (a =<br />

ASCII)<br />

char *inet_ntoa(struct in_addr in);<br />

int inet_aton(const char *cp, struct<br />

in_addr *inp); // Not avail for Windows<br />

inet_addr(): Predates inet_aton() but<br />

available under Windows<br />

in_addr_t inet_addr(const char *cp);<br />

• Deprecated because it can return -1 either<br />

because of error or successful conversion of<br />

broadcast address 255.255.255.255 = (int32) -1<br />

Copyright © 06/04/11, Tim Moors


20<br />

Using names<br />

Textual names are even more “presentable” than addresses<br />

(e.g. example.com vs 208.77.188.166)<br />

• See lecture on Domain Name System (DNS)<br />

• <strong>Socket</strong>s names services (e.g. “http”=80) as well as devices;<br />

ignore service name by passing NULL for serv(ice)<br />

int getaddrinfo(const char *node, const char<br />

*service, const struct addrinfo *hints, struct<br />

addrinfo **res);<br />

struct addrinfo {<br />

• Maps name to address<br />

• hints: e.g. to request IPv4 or IPv6<br />

• Release res with freeaddrinfo()<br />

int getnameinfo(<br />

const struct sockaddr *sa,<br />

socklen_t salen,<br />

int ai_flags; e.g. AI_NUMERICHOS<br />

int ai_family; //\<br />

int ai_socktype; //|-ala socket()<br />

int ai_protocol; ///<br />

size_t ai_addrlen;<br />

struct sockaddr *ai_addr;<br />

char *ai_canonname; Canonical<br />

struct addrinfo *ai_next; // Allo<br />

char *host, size_t hostlen, };<br />

char *serv, size_t servlen, int flags);<br />

• Maps address to “name”<br />

• flags: e.g. NI_NUMERICHOST for printable addr not DNS name<br />

Before IPv6 and Windows XP, apps used gethostbyname() & gethostbyaddr()<br />

Copyright © 06/04/11, Tim Moors


21<br />

Broadcast and multicast<br />

Use SOCK_DGRAM to broadcast or multicast, since SOCK_STREAM<br />

connects to a single endpoint<br />

Broadcast needs deliberate setup (to protect against mistake in<br />

address spamming network), through “setsockopt(s,<br />

SOL_SOCKET, SO_BROADCAST”<br />

Multicast doesn’t spread everywhere.<br />

• Receivers must tell routers to add/drop membership:<br />

App tells OS which uses IGMP † to tell routers:<br />

ip_mreq † struct identifies multicast address & interface<br />

setsockopt(s, IPPROTO_IP, IP_ADD_MEMBERSHIP, (void*)&mreq,<br />

sizeof(mreq));<br />

Then IP_DROP_MEMBERSHIP when done<br />

• Sender may want to further limit propagation (TTL)<br />

“setsockopt(… IP_MULTICAST_TTL”<br />

• Control whether you receive what you send to a multicast<br />

group that you’re a member of with “setsockopt(…<br />

IP_MULTICAST_LOOP”<br />

† IPv6 uses ICMPv6, rather than IGMP, and ipv6_mreq<br />

Copyright © 06/04/11, Tim Moors


22<br />

Cleaning up<br />

close() tells the OS that the app is finished with a socket.<br />

• For TCP sockets, OS usually † closes connection by sending<br />

FIN.<br />

• Important for servers that create new socket when<br />

accept()ing each new client. Without closing, socket<br />

resources would get used up.<br />

• close() stops both send & recv to/from socket<br />

o Info in queue to be sent will still be sent<br />

SO_LINGER option => Whether TCP lingers after close() to<br />

ensure reliable transfer (e.g. wait for acks & retransmit)<br />

shutdown(SOCKET s, int how) is like close, but<br />

• allows specific types of access to be shut down<br />

how: SHUT_RD, SHUT_WR, SHUT_RDWR<br />

e.g. client stops writing to indicate end of request, but waits<br />

to read reply.<br />

• Forces closure<br />

o close() maintains socket if still used by another process, server may<br />

fork() after accept() & one process close()s but other hasn’t finished<br />

Copyright © 06/04/11, Tim Moors


23<br />

Summary (for reference)<br />

SOCKET socket(int socket_family, int socket_type, int protocol);<br />

int setsockopt(SOCKET s,int level,int optname,const void *optval,socklen_t optlen);<br />

ssize_t send(SOCKET s,const void *buf,size_t len,int flags);<br />

ssize_t recv(SOCKET s, void *buf, size_t len, int flags);<br />

ssize_t sendto(SOCKET s,const void *buf,size_t len,int flags,const struct sockaddr<br />

*to,socklen_t tolen);<br />

ssize_t recvfrom(SOCKET s,void *buf,size_t len,int flags,struct sockaddr<br />

*from,socklen_t *fromlen);<br />

int bind(SOCKET s, const struct sockaddr *my_addr, socklen_t addrlen);<br />

int connect(SOCKET s, const struct sockaddr *serv_addr, socklen_t addrlen);<br />

int listen(SOCKET s, int backlog);<br />

SOCKET accept(SOCKET s, struct sockaddr *addr, socklen_t *addrlen);<br />

htonl(), ntohl(), htons(), ntohs()<br />

const char *inet_ntop(int af, const void *src, char *dst, socklen_t cnt);<br />

int inet_pton(int af, const char *src, void *dst);<br />

int getaddrinfo(const char *node, const char *service, const struct addrinfo *hints, struct<br />

addrinfo **res);<br />

int getnameinfo(const struct sockaddr *sa, socklen_t salen, char *host, size_t hostlen,<br />

char *serv, size_t servlen, int flags);<br />

close()<br />

shutdown(SOCKET s, int how)<br />

struct sockaddr {<br />

unsigned short sa_family;<br />

char sa_data[14];<br />

};<br />

Copyright © 06/04/11, Tim Moors<br />

struct sockaddr_in{<br />

short sin_family;<br />

unsigned short sin_port;<br />

struct in_addr sin_addr;<br />

char sin_zero[8];<br />

};


24<br />

Links<br />

• <strong>Socket</strong> options give control of TCP/IP,<br />

e.g. Nagle algorithm, IP TTL, IGMP etc<br />

• Framing: Apps that use SOCK_STREAM<br />

must often frame their data; e.g. using<br />

length fields<br />

Copyright © 06/04/11, Tim Moors

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!