Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
2380
votes
47 answers

How to concatenate text from multiple rows into a single text string in SQL Server

Consider a database table holding names, with three rows: Peter Paul Mary Is there an easy way to turn this into a single string of Peter, Paul, Mary?
JohnnyM
  • 28,758
  • 10
  • 38
  • 37
1417
votes
40 answers

How can I output MySQL query results in CSV format?

Is there an easy way to run a MySQL query from the Linux command line and output the results in CSV format? Here's what I'm doing now: mysql -u uid -ppwd -D dbname << EOQ | sed -e 's/ /,/g' | tee list.csv select id, concat("\"",name,"\"") as…
MCS
  • 22,113
  • 20
  • 62
  • 76
1105
votes
20 answers

Save PL/pgSQL output from PostgreSQL to a CSV file

What is the easiest way to save PL/pgSQL output from a PostgreSQL database to a CSV file? I'm using PostgreSQL 8.4 with pgAdmin III and PSQL plugin where I run queries from.
Hoff
  • 38,776
  • 17
  • 74
  • 99
1076
votes
10 answers

Writing a pandas DataFrame to CSV file

I have a dataframe in pandas which I would like to write to a CSV file. I am doing this using: df.to_csv('out.csv') And getting the following error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u03b1' in position 20: ordinal not in…
user7289
  • 32,560
  • 28
  • 71
  • 88
882
votes
19 answers

How can I convert a comma-separated string to an array?

I have a comma-separated string that I want to convert into an array, so I can loop through it. Is there anything built-in to do this? For example, I have this string var str =…
Blankman
  • 259,732
  • 324
  • 769
  • 1,199
826
votes
7 answers

Remove trailing delimiting character from a delimited string

What is fastest way to remove the last character from a string? I have a string like a,b,c,d,e, I would like to remove the last ',' and get the remaining string back: OUTPUT: a,b,c,d,e What is the fastest way to do this?
I-M-JM
  • 15,732
  • 26
  • 77
  • 103
779
votes
13 answers

Dump a NumPy array into a csv file

How do I dump a 2D NumPy array into a csv file in a human-readable format?
Dexter
  • 11,311
  • 11
  • 45
  • 61
776
votes
21 answers

How to import CSV file data into a PostgreSQL table

How can I write a stored procedure that imports data from a CSV file and populates the table?
vardhan
  • 7,769
  • 3
  • 16
  • 3
771
votes
31 answers

How to export JavaScript array info to csv (on client side)?

I know there are lot of questions of this nature but I need to do this using JavaScript. I am using Dojo 1.8 and have all the attribute info in array, which looks like this: [["name1", "city_name1", ...]["name2", "city_name2", ...]] Any idea how I…
Sam007
  • 8,397
  • 4
  • 24
  • 34
751
votes
20 answers

Import multiple CSV files into pandas and concatenate into one DataFrame

I would like to read several CSV files from a directory into pandas and concatenate them into one big DataFrame. I have not been able to figure it out though. Here is what I have so far: import glob import pandas as pd # Get data file names path =…
jonas
  • 13,559
  • 22
  • 57
  • 75
718
votes
6 answers

How to avoid pandas creating an index in a saved csv

I am trying to save a csv to a folder after making some edits to the file. Every time I use pd.to_csv('C:/Path of file.csv') the csv file has a separate column of indexes. I want to avoid printing the index to csv. I tried: pd.read_csv('C:/Path to…
Alexis
  • 8,531
  • 5
  • 19
  • 21
706
votes
11 answers

CSV file written with Python has blank lines between each row

import csv with open('thefile.csv', 'rb') as f: data = list(csv.reader(f)) import collections counter = collections.defaultdict(int) for row in data: counter[row[10]] += 1 with open('/pythonwork/thefile_subset11.csv', 'w') as…
Alex Gordon
  • 57,446
  • 287
  • 670
  • 1,062
704
votes
12 answers

HTML Input="file" Accept Attribute File Type (CSV)

I have a file upload object on my page: with the following excel files on my desktop: file1.xlsx file1.xls file.csv I want the file upload to ONLY show .xlsx, .xls, & .csv files. Using the accept attribute,…
Dom
  • 38,906
  • 12
  • 52
  • 81
683
votes
25 answers

UnicodeDecodeError when reading CSV file in Pandas

I'm running a program which is processing 30,000 similar files. A random number of them are stopping and producing this error... File "C:\Importer\src\dfman\importer.py", line 26, in import_chr data = pd.read_csv(filepath, names=fields) File…
TravisVOX
  • 20,342
  • 13
  • 37
  • 41
665
votes
49 answers

Python Pandas Error tokenizing data

I'm trying to use pandas to manipulate a .csv file but I get this error: pandas.parser.CParserError: Error tokenizing data. C error: Expected 2 fields in line 3, saw 12 I have tried to read the pandas docs, but found nothing. My code is…
abuteau
  • 6,963
  • 4
  • 16
  • 20
1
2 3
99 100