Consider a DataFrame with a lot of columns and we need all of them except for one. In this short how-to article, we will learn a convenient way of selecting all columns except for one.
Pandas
The loc method is used for selecting rows and columns by their labels. We will write a condition in the loc method using the columns method and the name of the unwanted column.
df.loc[:, df.columns != "f3"]PySpark
We can use a list comprehension in the select function to create a list of the desired columns.
df.select([col for col in df.columns if col != "f2"])The expression inside the select function is a list comprehension that creates a list with all the column names if it is not “f2”.
It is important to note that another way of solving this task is to drop the undesired column. Both Pandas and PySpark have a drop function to do this operation.
This question is also being asked as:
- Retrieve DataFrame of all but one specified column.
