A polygon is defined as a concave polygon, when one or more interior angles are greater than 180°.  In simple words, a polygon which is not convex is a concave polygon.

In convex polygon we will find the vertex pointing outside the polygon, whereas in concave polygon the vertex has been 'pushed in', that is vertex point towards the interior of the polygon.

Concave Polygon
Here are few properties of a Concave Polygon:

1. If a line is drawn through a concave polygon, it can intersect the polygon into more than two parts as shown in figure below,

Concave Polygon Line Intersection
2. Few diagonals of a concave polygon lie outside the polygon. In figure below, you see the diagonal at the top is outside the polygon's interior space.

Concave Polygons

Regular concave polygon are the ones which has equal sides and equal interior angles. But a concave polygon has one or more interior angles which is greater than 180°. So, we can say that a concave polygon can never be regular polygon.
An irregular polygon is the one which has any length of sides and with any measure of interior angles. Polygons can be convex or concave, but all the concave polygons are irregular as they are not equal with their interior angles and sides.
Since each side and each interior angle is different in concave polygon, we do not have any particular formula to find its area. But we can split the concave polygon into other shapes and then find its area. 

Say, if you have an concave polygon (as shown in the figure below) which can be divided into two triangles, then you can first calculate the area's of two triangles and add their areas to find the area of given concave polygon.
Area of Concave Polygon

Solved Example

Question: Find the area for the concave polygon given:
Concave Polygon Example
Here we can divide the given polygon into two parts, square and rectangle.
Example of Concave Polygon

Let us find the area of square first

Area of Square = side $\times$ side

Side of square = 6 (given)

$\therefore$ Area = 36 sq units.

Now, lets find the area of the rectangle

Area of the rectangle = length $\times$ width

Length = 26

width = 8

Area = 26 $\times$ 8

$\therefore$ Area = 208 sq units

So, the total area of the concave polygon = Area of square + Area of rectangle.

The total area = 36 + 208

The total area of concave polygon = 244 sq units.