Regarding Google Consent Mode and Server
Regarding Google Consent Mode and Server Side Requests. If the Consent mode is stored in local storage how does Zaraz know what to send Google via events fired through the events API as the information will not be available within the cookies passed to Cloudflare via those API calls?
The reason I ask is that we setup up Google Analytics with Consent mode via Zaraz a few weeks ago and we have been pinged by Google's email systems saying that we have still not setup consent mode for this site. We are also unable to verify that consent mode is setup on the Google Ads/Analytics side. As all the verification logic seems to require either the consent information is sent directly to Google Ads (We send the data via Google Analytics) or we are using the Tag Assistant (which doesn't work with Cloudflare).
Is there a way for us to verify that Google is receiving the consent status from both our client side and servside sent events? (We have identified that the gcd parameter is being sent correctly from client side events through the debugger, but the debugger only shows client side events)
I am thinking that our server side events are not including the consent mode parameters which is why Google is still flagging us as non-compliant?
73 Replies
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
by server side requests I mean requests we are sending to the Events API from our server not Zaraz is sending the third parties
The Zaraz debugger does not show requests sent from our server directly to Zaraz AFAIK?
Cloudflare Docs
HTTP Events API · Cloudflare Zaraz docs
The Zaraz HTTP Events API allows you to send information to Zaraz from places that cannot run the Web API, such as your server or your mobile app. It …
as far as I can tell there is no way to pass google consent mode data to the HTTP Events API
as we do not have access to the localstorage in the browser at the time we send the event to the HTTP events API
We could manually set this in a cookie so that we can include the information with our request to the HTTP events API but as far as I am aware there is no documentation or likely support for pulling google consent mode data from a cookie?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
OK thats awesome thanks
I assume that takes the same object as zaraz.set:
Is there any way to debug the google consent mode values sent by the HTTP Events API?
We are a little blind currently to information sent to third parties from events initiated by the HTTP Events API.
It would be cool to get a debug log system like the Worker Real Time Logs system, I realise thats a big ask though.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
you mean the debug mode detailed here:
https://developers.cloudflare.com/zaraz/web-api/debug-mode
Cloudflare Docs
Debug mode · Cloudflare Zaraz docs
Zaraz offers a debug mode to troubleshoot the events and triggers systems. To activate debug mode you need to create a special debug cookie …
I am not sure how that impacts requests made via the HTTP Events API?
As this sets a cookie and the browser cookies are sent via the the HTTP Events API we certainly will have sent HTTP Events with the Zaraz debug cookie but the Zaraz debugger client side has never displayed details of events fired via the HTTP Events API.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
hmm, OK that's interesting.
If the execute statements are designed to push information into the zaraz debugger then that should be working on our side as the responses are handled and passed through to the browser where required, I will dump the requests and outputs directly from the zaraz HTTP Event calls into our logging system and see whats going on. Thanks.
@yo'av I can confirm that adding the debug cookie to system.cookies on the HTTP Events API does not result in any additional information being added into the execute section of the returned response, in fact that section is entirely empty. We have data in the fetch and the cookies section of the returned response though.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Hello @yo'av! Do you mean that in the tool configuration for Google Ads, we need to enable the consent mode in the settings? Is that compliant with Consent Mode v2? The text after the field only refers to v1. Also, what should I enter in the input field, considering that the consent implementation is a tool in Zaraz that updates the variables for Consent Mode v2?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
You were refering to that (attached image)?
So, if I enable that option in the settings then in the custom script, which is a tool in Zaraz, does it make sense to keep the initial set (google_consent_default)? If I remove it, will everything function as before?
<script>
var strAnalytics = zaraz.consent.get('xxxxxx') ? 'granted' : 'denied';
var strMarketing = zaraz.consent.get('xxxxxx') ? 'granted' : 'denied';
zaraz.set("google_consent_default", {
'ad_storage' : strMarketing,
'ad_user_data' : strMarketing,
'ad_personalization' : strMarketing,
'analytics_storage' : strAnalytics
});
document.addEventListener("zarazConsentChoicesUpdated", () => {
strAnalytics = zaraz.consent.get('xxxxxx') ? 'granted' : 'denied';
strMarketing = zaraz.consent.get('xxxxx') ? 'granted' : 'denied';
zaraz.set("google_consent_update", {
'ad_storage' : strMarketing,
'ad_user_data' : strMarketing,
'ad_personalization' : strMarketing,
'analytics_storage' : strAnalytics
});
});
</script>
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
I have followed the guidelines and believe the implementation is correct. In the attached image, you can see that
__zarazGoogleConsentV2String is being generated and updated correctly, as shown by the Zaraz debugger. However, in Google Analytics' DebugView, the non_personalized_ads value is constantly set to 1 (when it should be 0), and in Google Ads, the consent mode appears to be inactive. Do you think the problem lies outside of Zaraz?
Thanks for the response I will get our logic amended to initiate debug mode on the HTTP Events API as shown.
It might be worth adding this as a footnote on the HTTP Events API page as its not clear.
OK I have got debug mode working for server side events and its only made this murky lagoon which is Zaraz and Google Consent mode even murkier
It told me for an add to basket event it was sending this request:
Which does have google consent information in it but not in the format I was expecting.
I was expecting a gcd parameter like other requests to https://www.google-analytics.com/g/collect
However, I decided to verify it against what Zaraz would send if we initiated the add to basket from the client side.
Doing so returns a Google Analytics URL like this:
Which is missing both the ep.google_consent_update parameters as well as the gcd parameters
Also when sent client side, the request includes a dt parameter which I believe should be the page title, server side it doesn't include this parameter and obviously the screen size is incorrect on the server side request.
I have zero idea if I should be doing this or not, however adding a custom property to the google zaraz settings which sets the gcd parameter to the contents of the event parameter __zarazGoogleConsentV2String seems to just fix the problem......
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
A curl request which exhibits this behaviour:
The only thing I actually removed from this code is our debug cookie contents and the URL I am posting too
Attached an image with the GA4 config on the Zaraz endpoint the above code was sent too.
And then the Google URL which is returned in the zaraz._1 call:
This has had a few parameters replaced with asterisks
The body data from the request formatted:
Is there an update on this @yo'av It would be good to know if this is an us problem or a cloud flare problem.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
we are not using Google Ads within Zaraz
This is for the Google Analytics Zaraz module
do you realize Google Ads runs client side and so calling it over the HTTP API doesn't make much sense?We are not using Google Ads, we are using Google Analytics. Our Google Analytics and our Google Ads accounts are connected to each other. Our conversion events within Google Ads are pulled from Google Analytics and are not required to be fired on the website itself. We know that this works fine (We track hundreds of conversions per day through Zaraz HTTP Events API ecommerce conversions into our Google Analytics accounts and then into our Google Ads accounts.) What we have noticed is that after we thought we had fully implemented Google Consent Mode. We kept getting notifications to tell us that Google Ads was not happy with our consent signals. We had already confirmed that Consent Signals were being received by Google Analytics Via the Google Analytics dashboard, however Google Analytics does not tell us that some requests were being submitted without consent signals. We began to suspect the HTTP Events API was not sending consent signals after repeated emails from Google telling us that we would be getting audited for lack of consent signals weeks after having implemented them fully on our sites.
what's your goal with even trying this though?My goal is that we submit valid consent mode information to Google Analytics which can then be passed on to Google Ads. There's no information anywhere to indicate that this shouldn't work when data is submitted via the HTTP Events API? Are you saying that the HTTP Events API is unable to handle some of Zaraz's third party tools correctly? If so, is there any documentation as to which tools it does or does not work with? We aren't really trying this. This is already in production on 2 out of 3 of our ecommerce sites and is due to go live next week on the third our largest ecommerce site. One of the reasons we chose Zaraz for this functionality was because of its support for sending information directly to Google Analytics via the HTTP Events API. This allows us to send ecommerce related events from our queue server which handles jobs related to ecommerce order handling. (The queue server puts the results of these requests back into the session data for the particular customer the order came from. If the customer makes further requests on our site the appropriate client side code will be executed to handle the response. This obviously does not guarantee 100% of the time these requests will be fired as the customer may not make another request to our site after the queue server has processed the ecommerce order events. This is a fairly low chance though due to the way our site handle ecommerce orders.
are you taking the instructions from the fetch array and passing it to the browser afterwards to run the requests?Yes, we handle all the results returned from the HTTP events API as requested by the Events API response.
We are trying to migrate away from as much client side logic as possible. Part of this journey was switching to Zaraz but there were a lot of other elements to it.
This work has allowed us to completely reverse our core web vitals scores for the 2 sites we have already migrated.
My team and I really do like Zaraz we think it makes some really complex problems easy to solve and allows us to focus on what should be the bigger pieces of work for us such as improving our sites.
We did review Server side Tag Manager and using the Google Analytics Measurement Protocol directly, however neither quite solved our problems like Zaraz did.
Server side tag manager is frustratingly complex to setup and manage and the majority of the documentation assumes you are configuring it within Google Cloud which we do not use.
Measurement protocol would have helped, especially with our requirement to send server side ecommerce events. However it only really solved part of our problem as it would have left the client's browser riddled with tracking services and it would have required us to handle this one specific analytics provider differently from our other providers.
Our only concerns with Zaraz having implemented it now on our sites are that the documentation is really light and sometimes it doesn't really do what you would expect it to do.
I understand that you created Zaraz and then merged it into Cloudflare with a current pricing model which is extremely fair, I am appreciative of that. I assume that without Cloudflare's support Zaraz as a third party service would probably have needed to be priced very differently and likely would not have been achievable for us.
This is probably fairly small potatoes in the grand scheme of the Cloudflare world but our 2 production sites are totalling around 125k events per week in Zaraz, the third site we are about to launch will probably add a further 200k events per week on its own.
We are aiming to roll out additional sites at around 2-5 per year moving forward as our business expands internationally.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Thanks, if its possible to message here when it is working then I can get it tested and remove our hack, otherwise I will check back in Monday next week.
I demo'd Zaraz to our third party marketing company today who operate primarily around GTM
They said that Zaraz looked really capable and simple to use from their point of view compared to their experience with GTM.
I live setup a bunch of triggers and actions and showed how they can be used to detect certain situations (customers click a CTA, interact with our forms, etc) and how this information could be forwarded to Google Analytics or our other Analytics tools. Using the Zaraz debugger to show the events firing and how the system distributed the events to the third parties and then showing those events arriving at the third party providers.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
I have added the "gcd" custom field in the Zaraz Google Ads tool, but in Google Ads, the diagnostics for the consent mode are still not green. In Analytics, the consent signals are automatically sending user consent preferences correctly its green long ago and those two entities are long ago linked. I am totally confused.
The hacky solution I provided was specific to Google analytics for sending consent mode data when requests are made via the zaraz http events API. If that situation does not match yours then the hack I identified will not work for you.
From yesterday to today, there has been no change in the status. Have you implemented Google Ads with zaraz before? If so, did you face any issues, or was it streamlined and working correctly?
No, nothing I have written about has anything to do with using Google ads with zaraz. It's all about using Google analytics and the issues I have raised are very specific to the http events API. If you aren't using these then my information won't apply to your situation.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
In the Google Ads tool the Conversion ID the Conversion Label and the Conversion Linker must be enabled only in the conversion action, on tool settings or on both?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Despite the fact that on Google Analytics the consent signals are ok (the indicators in the data stream about the consent are green) in Google Ads still have that warning (attached). In zaraz debugger on both zaraz tools (analytics / ads) the info about the signals is passed. I'm missing any configuration?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
To be more specific, for instance in an action called "add to cart" in debugger I see the following info (attached image). Is there any other info that should be displayed here? Is that correct? Also why i dont see server action called (like for the analytics tool)?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
For Google Ads only, "Action called" I see in the Zaraz debugger and not server-side network request. Network requests I see going only to Google Analytics, having in the resource URL the "gcd" parameter. Could the lack of the Google Ads server-side network request be the problem?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
That's the body of the request, contains the "gcd".
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
I'm not sure if is totally good. In Analytics is good. In Ads I'm not sure because I have one warning (attachment) and one diagnostic issue (attachment). Those 2 and especially the warning is my concern.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Is it possible that there are more than one requests? What I mean is, is there another request for Google Analytics or another request for Google Ads? I see only one request here in my implementation.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Can you provide a screenshot from the zaraz debugger with a clear view of the contents especially something including Google Ads; Thank you!
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Of course, all the requests are appearing in the Zaraz debugger. In the network, I see the gcd param (previews message) that hits a google.gr URL, and I think that request is mainly for Google Ads. Am I correct?
In the debugger, I see the above attached, and except for the action called, I don't see anything more for Google Ads. Is that also correct, or is there something else that should be presented in the debugger?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Amusingly when you reach out to Google Ads they say roughly the same thing because of the use of Zaraz. Although Google support services are absolutely awful if you have any issue with regards to sending data to their services they just tell you to follow the google tag manager install guide and say that's all the support they can provide.
They won't even touch debugging these situations because the requests are sent by Zaraz not one of their libraries.
Worse neither Google Ads nor Google Analytics provide any debugging tools for Consent mode information (Google Analytics debug view does not show consent mode information sent as part of the specific requests as far as I can tell). The only thing either of them provide are flags to indicate if it is working or not. Google Analytics flags just say "consent signals active". I am told that there are flags in the Google Ads dashboard but I have never seen them before as we are yet to get consent mode information into Google Ads successfully and they apparently only show up once consent mode is working correctly. I am not even sure if those flags actually work when data is forwarded from Analytics to Ads, their documentation is not clear at all.
ive never seen a "client side network request" in the zaraz debugger, so I dont think that logic is functional currently.
I would imagine that the MS UET library should have client side network request events in the debugger, as both Facebook and Google go server-side and the server-side requests are shown correctly.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
That is precisely the issue.
It appears that Google may have made some changes, as the consent signals are no longer being recognized and there are no indicators showing the status.
I contacted Google and their representatives informed me that our implementation was not correct. They were unable to provide further assistance as we do not use Tag Manager (which I find unprofessional, but as they are the main product, all others must comply with them).
@yo'av Can Cloudflare / Zaraz offer assistance in addressing these issues? The inconsistencies with the consent signal have had a significant impact on our company's strategies and have greatly affected the performance of our Google Ads campaign. So far, we have only experienced negative effects such as fewer purchases, lower income, and difficulties with targeting our campaign audience.
I am unsure if I am responsible for maintaining Zaraz and its tools, or if I should revert back to using Cookiebot and Tag Manager. If Cloudflare does not plan on addressing these issues, it will be challenging to troubleshoot, verify, and upkeep.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Thanks, I will test on our staging system and verify it's working as expected now.
@yo'av does it take some time to deploy on your side or would you expect it to be working already,
No gcd parameter
Test from a few moments ago (ignore &ep.dsfrgsdfg=asdfasdf I added this to verify the zaraz config was synced)
this was on our styaging system
event sent through HTTP Events API
with client google_consent_update parameter
I have also tried sending add to basket events from client side via Zaraz and the same issue occurs, no gcd parameter:
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
sure
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
@Thanos I cannot help with your Google Ads Consent logic, please don't add me as a friend to try to DM. I am not a Cloudflare or Google support person.
You maybe want to take a look at your consent mode settings though as your gcd string indicates that you are granting consent both by default and after acceptance.
Granting consent by default may be getting your gcd parameter flagged as incorrect. You cannot grant consent by default and be in alignment with EU laws as far as I am aware.
The gcd parameter is broken down like this:
there are other countries/regions where it may be possible to assume consent unless it is actively opted out of, but not in the EU
which likely explains why the option exists
@yo'av
Re-tested just now and there does not seem to be any difference, exactly the same results as earlier today and on the 2nd of August.
I am slightly concerned about the changelog shown here though: https://discord.com/channels/595317990191398933/1040420029080018945/1272547597994950687
As we believe that the issue is with the Zaraz ecommerce module not the HTTP Events API
This is due to the fact Zaraz does not send consent signals (gcd param) for google analytics requests on either client side or HTTP Events API side for ecommerce events (I use add to basket/cart for my testing).
Has a different problem potentially been identified and fixed, different to the one I identified previously? We wouldn't send page view events through the HTTP Events API and we don't use any custom events currently only pageview(client side) + the ecommerce events(mix of client side and server side).
"Order Completed" is the event we have particularly major issues with as this is marked as a key event in GA4 which is then passed to Google Ads and then Ads is telling us that we are not sending them consent signals.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
The issue has nothing to do with e-commerce. why do you think it does?Because the issue happens when you send an ecommerce event from the client side. confirmed it happens for "product viewed" ecommerc event also @yo'av I have sent a friend request as I can't send DMs unless we are friends on Discord.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
OK either way, we havent managed to trigger any ecommerce event with the gcd parameter but it works fine for the pageview event.
but those are the only events we are using so I cant comment on anything else.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
can you confirm that non ecommerce events on the http api endpoint are working well for you now?We have never sent non-ecommerce events via the HTTP api and have never raised that as an issue. I cant confirm that because we simply dont do it. I have no way of replicating that situation in our environment we originally raised this as a HTTP Events API issue because we noticed that HTTP Events API requests never included the gcd parameter. But that was entirely because no ecommerce event ever includes the gcd parameter, not because it was anything to do with the HTTP Events API
I never see "client side network request" in the zaraz debugger, has to do with my website implementation being SSR (I use remix.js for development)?
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Thanks @yo'av I will be able to test this on our side in about an hour.
Sorry for the delay, I have confirmed that both client side network requests in the debugger and GA4 ecommerce events sending the gcd parameter are working as expected.
The only quirk is that the debugger now displays some slightly odd info as shown in the attached screenshot.
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View