[PATCH] Explicit null dereferenced (src/backend/access/heap/heaptoast.c)

Started by Ranier Vilelaover 5 years ago3 messages
#1Ranier Vilela
ranier.vf@gmail.com
1 attachment(s)

Hi,

Per Coverity.

When "Prepare for toasting", it is necessary to turn off the flag
TOAST_NEEDS_DELETE_OLD,
if there is no need to delete external values from the old tuple, otherwise,
there are dereference NULL at toast_helper.c (on toast_tuple_cleanup
function).

regards,
Ranier Vilela

Attachments:

fix_null_dereference_heaptoast.patchapplication/octet-stream; name=fix_null_dereference_heaptoast.patchDownload
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 584f101dd9..a12bd194e9 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -147,6 +147,7 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
 	{
 		ttc.ttc_oldvalues = NULL;
 		ttc.ttc_oldisnull = NULL;
+        ttc->ttc_flags &= ~TOAST_NEEDS_DELETE_OLD;
 	}
 	else
 	{
#2Noname
gkokolatos@pm.me
In reply to: Ranier Vilela (#1)
Re: [PATCH] Explicit null dereferenced (src/backend/access/heap/heaptoast.c)

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, 28 August 2020 03:22, Ranier Vilela <ranier.vf@gmail.com> wrote:

Hi,

Per Coverity.

When "Prepare for toasting", it is necessary to turn off the flag TOAST_NEEDS_DELETE_OLD,
if there is no need to delete external values from the old tuple, otherwise,
there are dereference NULL at toast_helper.c (on toast_tuple_cleanup function).

Excuse my ignorance, isn't this a false positive?

Regardless right after prepare for toasting, a call to toast_tuple_init is made which will explicitly and unconditionally set ttc_flags to zero so the flag bit set in the patch will be erased anyways. This patch may make coverity happy but does not really change anything in the behaviour of the code.

Furthermore, in the same function, toast_tuple_init, the flag is set to TOAST_NEEDS_DELETE_OLD after the old value is actually inspected and found to not be null, be stored on disk and to be different than the new value. To my understanding, this seems to be correct.

Can you please explain to me what I am missing?

//Georgios

Show quoted text

regards,
Ranier Vilela

#3Ranier Vilela
ranier.vf@gmail.com
In reply to: Noname (#2)
1 attachment(s)
Re: [PATCH] Explicit null dereferenced (src/backend/access/heap/heaptoast.c)

Em sex., 28 de ago. de 2020 às 04:45, <gkokolatos@pm.me> escreveu:

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, 28 August 2020 03:22, Ranier Vilela <ranier.vf@gmail.com>
wrote:

Hi,

Per Coverity.

When "Prepare for toasting", it is necessary to turn off the flag

TOAST_NEEDS_DELETE_OLD,

if there is no need to delete external values from the old tuple,

otherwise,

there are dereference NULL at toast_helper.c (on toast_tuple_cleanup

function).

Excuse my ignorance, isn't this a false positive?

Yes, you're right.

Coverity fails with &.

if (oldtup == NULL)
147 {

3. assign_zero: Assigning: ttc.ttc_oldvalues = NULL.
148 ttc.ttc_oldvalues = NULL;
149 ttc.ttc_oldisnull = NULL;

4. Falling through to end of if statement.
150 }
151 else
152 {
153 ttc.ttc_oldvalues = toast_oldvalues;
154 ttc.ttc_oldisnull = toast_oldisnull;
155 }
156 ttc.ttc_attr = toast_attr;
157 toast_tuple_init(&ttc); // Coverity ignores the call completely
here.

toast_tuple_init, solves the bug, because reset ttc->flags.

Regardless right after prepare for toasting, a call to toast_tuple_init is
made which will explicitly and unconditionally set ttc_flags to zero so the
flag bit set in the patch will be erased anyways. This patch may make
coverity happy but does not really change anything in the behaviour of the
code.

That's right, the patch doesn't change anything.

Furthermore, in the same function, toast_tuple_init, the flag is set to
TOAST_NEEDS_DELETE_OLD after the old value is actually inspected and found
to not be null, be stored on disk and to be different than the new value.
To my understanding, this seems to be correct.

Very correct.

Thanks for taking a look here.

You could take a look at the attached patch,
would it be an improvement?
toast_tuple_init, it seems to me that it can be improved.
ttc->ttc_oldvalues is constant, and it could be unlooping?

regards,
Ranier Vilela

Attachments:

unloop_toast_tuple_init.patchapplication/octet-stream; name=unloop_toast_tuple_init.patchDownload
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 584f101dd9..1c57cc515e 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -133,8 +133,6 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
 
 	Assert(numAttrs <= MaxHeapAttributeNumber);
 	heap_deform_tuple(newtup, tupleDesc, toast_values, toast_isnull);
-	if (oldtup != NULL)
-		heap_deform_tuple(oldtup, tupleDesc, toast_oldvalues, toast_oldisnull);
 
 	/* ----------
 	 * Prepare for toasting
@@ -143,6 +141,7 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
 	ttc.ttc_rel = rel;
 	ttc.ttc_values = toast_values;
 	ttc.ttc_isnull = toast_isnull;
+	ttc.ttc_attr = toast_attr;
 	if (oldtup == NULL)
 	{
 		ttc.ttc_oldvalues = NULL;
@@ -150,10 +149,10 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
 	}
 	else
 	{
+		heap_deform_tuple(oldtup, tupleDesc, toast_oldvalues, toast_oldisnull);
 		ttc.ttc_oldvalues = toast_oldvalues;
 		ttc.ttc_oldisnull = toast_oldisnull;
 	}
-	ttc.ttc_attr = toast_attr;
 	toast_tuple_init(&ttc);
 
 	/* ----------
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..ca5d6f5fc6 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -48,62 +48,11 @@ toast_tuple_init(ToastTupleContext *ttc)
 
 	for (i = 0; i < numAttrs; i++)
 	{
-		Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
-		struct varlena *old_value;
-		struct varlena *new_value;
+		Form_pg_attribute att;
 
 		ttc->ttc_attr[i].tai_colflags = 0;
 		ttc->ttc_attr[i].tai_oldexternal = NULL;
 
-		if (ttc->ttc_oldvalues != NULL)
-		{
-			/*
-			 * For UPDATE get the old and new values of this attribute
-			 */
-			old_value =
-				(struct varlena *) DatumGetPointer(ttc->ttc_oldvalues[i]);
-			new_value =
-				(struct varlena *) DatumGetPointer(ttc->ttc_values[i]);
-
-			/*
-			 * If the old value is stored on disk, check if it has changed so
-			 * we have to delete it later.
-			 */
-			if (att->attlen == -1 && !ttc->ttc_oldisnull[i] &&
-				VARATT_IS_EXTERNAL_ONDISK(old_value))
-			{
-				if (ttc->ttc_isnull[i] ||
-					!VARATT_IS_EXTERNAL_ONDISK(new_value) ||
-					memcmp((char *) old_value, (char *) new_value,
-						   VARSIZE_EXTERNAL(old_value)) != 0)
-				{
-					/*
-					 * The old external stored value isn't needed any more
-					 * after the update
-					 */
-					ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_DELETE_OLD;
-					ttc->ttc_flags |= TOAST_NEEDS_DELETE_OLD;
-				}
-				else
-				{
-					/*
-					 * This attribute isn't changed by this update so we reuse
-					 * the original reference to the old value in the new
-					 * tuple.
-					 */
-					ttc->ttc_attr[i].tai_colflags |= TOASTCOL_IGNORE;
-					continue;
-				}
-			}
-		}
-		else
-		{
-			/*
-			 * For INSERT simply get the new value
-			 */
-			new_value = (struct varlena *) DatumGetPointer(ttc->ttc_values[i]);
-		}
-
 		/*
 		 * Handle NULL attributes
 		 */
@@ -114,11 +63,19 @@ toast_tuple_init(ToastTupleContext *ttc)
 			continue;
 		}
 
+		att = TupleDescAttr(tupleDesc, i);
 		/*
 		 * Now look at varlena attributes
 		 */
 		if (att->attlen == -1)
 		{
+		    struct varlena *new_value;
+
+		    /*
+		     * For INSERT simply get the new value
+		     */
+		    new_value = (struct varlena *) DatumGetPointer(ttc->ttc_values[i]);
+			
 			/*
 			 * If the table's attribute says PLAIN always, force it so.
 			 */
@@ -158,6 +115,56 @@ toast_tuple_init(ToastTupleContext *ttc)
 			ttc->ttc_attr[i].tai_colflags |= TOASTCOL_IGNORE;
 		}
 	}
+
+    if (ttc->ttc_oldvalues != NULL)
+	{		
+	    for (i = 0; i < numAttrs; i++)
+	    {
+		    Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+		    struct varlena *old_value;
+
+		    /*
+		     * For UPDATE get the old value of this attribute
+		     */
+		    old_value =
+			    (struct varlena *) DatumGetPointer(ttc->ttc_oldvalues[i]);
+
+		    /*
+		     * If the old value is stored on disk, check if it has changed so
+		     * we have to delete it later.
+		     */
+		    if (att->attlen == -1 && !ttc->ttc_oldisnull[i] &&
+			    VARATT_IS_EXTERNAL_ONDISK(old_value))
+		    {
+		        struct varlena *new_value;
+
+		        new_value =
+			        (struct varlena *) DatumGetPointer(ttc->ttc_values[i]);
+
+			    if (ttc->ttc_isnull[i] ||
+				    !VARATT_IS_EXTERNAL_ONDISK(new_value) ||
+				     memcmp((char *) old_value, (char *) new_value,
+					       VARSIZE_EXTERNAL(old_value)) != 0)
+			    {
+				    /*
+				     * The old external stored value isn't needed any more
+				     * after the update
+				     */
+				    ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_DELETE_OLD;
+				    ttc->ttc_flags |= TOAST_NEEDS_DELETE_OLD;
+			    }
+			    else
+			    {
+				    /*
+				     * This attribute isn't changed by this update so we reuse
+				     * the original reference to the old value in the new
+				     * tuple.
+				     */
+				    ttc->ttc_attr[i].tai_colflags |= TOASTCOL_IGNORE;
+			    }
+		    }
+	    }
+	}
 }