File I/O Advanced

Atomicity and race conditions

• All system calls are executed atomically. By this, we mean that the kernel guarantees that all of the steps in a system call are completed as a single operation, without being interrupted by another process or thread.

• Atomicity is essential to the successful completion of some operations. In particular, it allows us to avoid race conditions

Example

if (lseek(fd, 0, SEEK_END) == -1)
    errExit("lseek");
if (write(fd, buf, len) != len)
    fatal("Partial/failed write");

• If the first process executing the code is interrupted between the lseek() and write() calls by a second process doing the same thing, then both processes will set their file offset to the same location before writing, and when the first process is rescheduled, it will overwrite the data already written by the second process (aka race conditions occurred).

File control operations: fcntl()

• The fcntl() system call performs a range of control operations on an open file descriptor.

#include <fcntl.h>
int fcntl(int fd, int op, ...);

// Return on success depends on operation, or –1 on error

Explain

• fd: file descriptor
• op: operation

Usage: duplicating a file descriptor

Operation	Description
F_DUPFD	Duplicate the file descriptor fd (!= dup2(2))
F_DUPFD_CLOEXEC	As for F_DUPFD, but add close-on-exec flag for the duplicate fd

Usage: File status flags

• Each open file description has certain associated status flags, initialized by open(2) and possibly modified by fcntl().
• Duplicated file descriptors (made with dup(2), fcntl(F_DUPFD), fork(2), etc.) refer to the same open file description, and thus share the same file status flags.

Operation	Description
F_GETFL	Return the file access mode and file status flags; arg is ignored
F_SETFL	Set the file status flags to the values specified by arg

• Example:

  int flags, accessMode;
  flags = fcntl(fd, F_GETFL); /* Third argument is not required */
  if (flags == -1)
      errExit("fcntl");
  
  // Test if the file was opened for synchronized writes
  if (flags & O_SYNC)
      printf("writes are synchronized\n");

• Using fcntl() to modify open file status flags is particularly useful in the following cases:

  • The file was not opened by the calling program, so that it had no control over the flags used in the open() call.
  • The file descriptor was obtained from a system call other than open(). Examples of such system calls are pipe().

• Example 2: modify the open file status flags.

int flags;
flags = fcntl(fd, F_GETFL);
if (flags == -1)
    errExit("fcntl");
flags |= O_APPEND;
if (fcntl(fd, F_SETFL, flags) == -1)
    errExit("fcntl");

Relationship between file descriptors and open files

** Note that file descriptor is a non-negative integer that act as an abstract to handle file or I/O resources 0,1,2 are taken by STDIN, STDOUT, and STDERR. **

• It is possible and useful to have multiple descriptors referring to the same open files. These file descriptors may be open in the same process or in different processes.

• For each process, the kernel maintains a table of open file descriptors. Each entry in this table records information about a single file descriptor, including:

  • a set of flags controlling the operation of the file descriptor (just 1 now: close-on-exec flag).
  • a reference to the open file description.

• The kernel maintains a system-wide table of all open file descriptions (aka open file handles). An open file description stores all information relating to an open file, including:

  • the current file offset (as updated by read() and write(), or explicitly modified using lseek());
  • status flags specified when opening the file (i.e., the flags argument to open());
  • the file access mode (read-only, write-only, or read-write, as specified in open());
  • settings relating to signal-driven I/O
  • a reference to the i-node table object for this file

• Each file system has a table of i-nodes for all files residing in the file system. The i-node for each file includes the following information:

  • file type (e.g., regular file, socket, or FIFO) and permissions
  • a pointer to a list of locks held on this file; and
  • various properties of the file, including its size and timestamps relating to different types of file operations.

  

  • In process A, descriptors 1 and 20 both refer to the same open file description This situation may arise as a result of a call to dup(), dup2(), or fcntl().
  • Descriptor 2 of process A and descriptor 2 of process B refer to a single open file description. This scenario could occur after a call to fork() or if 1 process passed and open descriptor to another process using socket.
  • Descriptor 0 of process A and descriptor 3 of process B refer to different open file descriptions, but that these descriptions refer to the same i-node table entry => each process independently called open() for the same file.

Duplicating file descriptors: dup(), dup2()

• Consider the following bash shell command:

./myscript > results.log 2>&1

• Syntax 2>&1 informs the shell that we wish to have standard error (file descriptor 2) redirected to the same place to which standard output (file descriptor 1) is being sent.
• The shell achieves the redirection of standard error by duplicating file descriptor 2 so that it refers to the same open file description as file descriptor 1.
• This can be achieved using the dup() and dup2() system calls.
• Note that it is not sufficient for the shell simply to open the results.log file twice: once on descriptor 1 and once on descriptor 2. One reason for this is that the two file descriptors would not share a file offset pointer, and hence could end up overwriting each other’s output

dup()

• The dup() call takes oldfd, an open file descriptor, and returns a new descriptor that refers to the same open file description. The new descriptor is guaranteed to be the lowest unused file descriptor.
• By default, the shell has opened file descriptors 0, 1 and 2. dup() will use 3 as the start.
• dup(fd) is equivalent to fcntl(filedes, F_DUPFD, 0)

#include <unistd.h>
int dup(int oldfd);

// Returns (new) file descriptor on success, or –1 on error

• Example

  #include <fcntl.h>
  #include <stdio.h>
  #include <sys/stat.h>
  #include <unistd.h>
  
  int main(int argc, char *argv[]) {
    int inputFd;
    inputFd = open(argv[1], O_RDONLY);
    printf("Input fd value: %d\n", inputFd);
  
    int copyFd = dup(inputFd);
    printf("Copy fd value: %d\n", copyFd);
  // 3
  // 4
  
  }

dup2()

• To get the file descriptor we want (not starting from 3), the dup2() system call makes a duplicate of the file descriptor given in oldfd using the descriptor number supplied in newfd

#include <unistd.h>
int dup2(int oldfd, int newfd);
// Returns (new) file descriptor on success, or –1 on error

dup3()

#define _GNU_SOURCE
#include <unistd.h>
int dup3(int oldfd, int newfd, int flags);

// Returns (new) file descriptor on success, or –1 on error

• The dup3() system call performs the same task as dup2(), but adds an additional argument, flags, that is a bit mask that modifies the behavior of the system call.
• Currently, dup3() supports one flag, O_CLOEXEC, which causes the kernel to enable the close-on-exec flag (FD_CLOEXEC) for the new file descriptor. This flag is useful for the same reasons as the open() O_CLOEXEC flag

File I/O at specified offset: pread() and pwrite()

• The pread() and pwrite() system calls operate just like read() and write(), except that the file I/O is performed at the location specified by offset, rather than at the current file offset. The file offset is left unchanged by these calls.
• Calling pread() is equivalent to atomically performing the following calls:

off_t orig;
orig = lseek(fd, 0, SEEK_CUR); /* Save current offset */
lseek(fd, offset, SEEK_SET);
s = read(fd, buf, len);
lseek(fd, orig, SEEK_SET);

Scatter-Gather I/O: readv() and writev()

Truncating a File: truncate() and ftruncate()

Nonblocking I/O

I/O on Large Files

The /dev/fd directory

• For each process, the kernel provides the special virtual directory /dev/fd. This directory contains filenames of the form /dev/fd/n, where n is a number corresponding to one of the open file descriptors for the process.
• Opening one of the files in the /dev/fd directory is equivalent to duplicating the corresponding file descriptor. Thus, the following statements are equivalent:

fd = open("/dev/fd/1", O_WRONLY);
fd = dup(1);

• /dev/fd is actually a symbolic link to the /proc/self/fd directory (current running process). The latter directory is a special case of the /proc/PID/fd directories, each of which contains symbolic links corresponding to all of the files held open by a process.

Creating temporary files

mkstemp()

• The mkstemp() function generates a unique filename based on a template supplied by the caller and opens the file, returning a file descriptor that can be used with I/O system calls

#include <stdlib.h>
int mkstemp(char *template);

// Returns file descriptor on success, or –1 on error

• The template argument takes the form of a pathname in which the last 6 characters must be XXXXXX.
• Example:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {
  int fd;
  char template[] = "/tmp/somestringXXXXXX";

  fd = mkstemp(template);
  if (fd == -1)
    fprintf(stderr, "mkstemp error\n");

  printf("Generated filename was: %s\n", template);
  unlink(template);

  // use file i/o system calls

  // close
  if (close(fd) == -1)
    fprintf(stderr, "close error\n");
}

tmp()

• The tmpfile() function creates a uniquely named temporary file that is opened for reading and writing. (The file is opened with the O_EXCL flag to guard against the unlikely possibility that another process has already created a file with the same name.)

#include <stdio.h>
FILE *tmpfile(void);

// Returns file pointer on success, or NULL on error