Read a large file line-by-line in Python


The fileinput module's input method reads in files one line at a time:

import fileinput

for line in fileinput.input(["somefile.txt"]):
	print(line)

BONUS: How to read a large CSV in chunks:

import pandas as pd

# `chunksize` specifies the number of rows in each chunk
chunksize = 10000

with pd.read_csv("big.csv", chunksize=chunksize) as reader:
  for chunk in reader:
    print(chunk)

Sources