How to Combine Two Text Columns in a DataFrame?

How to Merge Two Columns in a Dataframe Pandas and Pyspark

We sometimes need to create columns by combining two or more columns together. In this how-to article, we will learn how to combine two text columns in Pandas and PySpark DataFrames.

combine-two-text-columns-in-dataframe

Pandas

We can combine text columns with the “+” operator.

				
					df["full_name"] = df["first_name"] + " " + df["last_name"]
				
			

The expression in between is used for adding a space between the first and last names. Another way of combining text columns is aggregating columns by joining.

				
					df["full_name"] = df[["first_name","last_name"]].agg(" ".join, axis=1)
				
			

We can use both these methods to combine as many columns as needed. The only requirement is that the columns must be of object or string data type.

PySpark

We can use the concat function for this task.

				
					df = df.withColumn(
    "full_name", 
    F.concat("first_name", F.lit(" "), "last_name")
)
				
			

The lit function is used for adding the space between the first and last names.

This question is also being asked as:

  • Python combining two columns.

People have also asked for:

You may also like

Start Monitoring Your Models in Minutes