-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setting up a test battery #1271
Comments
reserved |
let's start with how to drive cromite from the outside. the choice is to use Appium + UiAutomator2 + chromedriver, driven by webdriver in nodejs, but I started out knowing nothing about all! first of all, I discovered that there are 2 versions of appium, the one officially documented in browserstack (but old, the v1) and one undocumented (v2) which, without knowing it, I used. In fact, I found differences that I could not explain, but which I only realised later, since most of the things I tried still worked. this made it difficult for me to understand, also because the driver of the old version was called wd while that of the new webdriver and at first I thought it was just the initials :) so I developed a solution with wd only realising later that I was doing it wrong :( Only after banging my head several times do I realise that:
everything communicates via http, between the client and server part of the component, with the sole exception of UiAutomator2 which drives android via adb. chromedriver also talks to chromium via a http socket tunnelled by adb. the various layers allow me different things
UiAutomator2 works flawlessly, chromedriver needs some hack, also because it is not officially supported by browserstack (and moreover not updated, being still at v123) and I need my solution to work on the emulator as on a real device. the problem is that to get chromedriver to work with chromium forks (all of them, the problem is not just cromite) you have to cheat some of its basic settings. an internet search did not directly bring a solution, because it seems that no one has ever succeeded (or maybe I haven't found it).
except in browserstack! that does not accept the passing of that information. i have asked their support, i have not currently received any technical response (if not look in the documentation :) (EDIT: 8/2/24 i finally received it a few weeks ago and they were very helpful. we agreed that since the request was specific to cromite and they had already fixed my side, no intervention from them was necessary) done that I haven't done anything yet :) but it's a first step and I got here. |
This comment was marked as off-topic.
This comment was marked as off-topic.
late by a few weeks, here you will find a summary of the results of creepjs acquired automatically via browserstack, I will put the code to do this file as soon as I clean it up. in the excel there are:
I need someone to help me draw conclusions. @PF4Public :) |
ah sorry, @abrahamjuliot I forgot about you! I think you already have this data, but I'd like you to participate in the discussion, if you have the time and desire. the topic is: what can i do with it and most importantly, how? I already have some ideas. |
for an initial analysis of the data i used power bi desktop, which, i admit, i do not know how to use very well. i find qlikview simpler and more complete, but unfortunately, it is not free. The element I first checked is what creepjs calls emojiiSet, verifying that indeed the patch returns the same result for each device: it did. the anomaly concerning line 4 and 2 emojiiSet for chrfont tests (section A in the image) is related to an error in the patch I produced, for now let's pretend those data do not exist. The totals at the bottom are also to be ignored, because it happened that some devices were not available during the test, while others crashed, allow me to be a little superficial at this stage. It is clear from the figure that:
with regard to the difference between cromite and chromium (2), it is due to the fact that emojii in Chromium are derived from play services, which, of course, are disabled in cromite ..continue.. |
If we consider the emojiiset-0 group, these are the results for the devices for the chromium test: (I'm sorry you can't see them all) these the same for the chrfonts1 test: it is immediately clear that the patch flattens all fonts, since the value for ..continue.. |
I then looked for which value would be most representative for comparison and found that
..continue.. |
at this point I focused on why some results are different, and I found this:
Basically, it appears that the value of I do not put the data from the chromium test because there are many, but the conclusion is the same. ..continue.. |
so I tried to make sense of the
I assume from that table that screen width is an important factor, more so than height. ..I'll stop for now.. ;) |
Nice findings. I'd guess device pixel ratio and maybe the screen width is giving the dimensions hardware entropy. It looks like the model, or brand, can be projected based on the screen and pixel ratio. Device scaling seems not to impact |
the data do not say so: different ratios produce equal results.
and even this seems strange to me, since the patch modifying the viewport is based on just that.
this is certain, I have already checked it. there are indications in another issue (which I can no longer find), where the history of git with regard to fonts was checked.
yes, I am aware of that. chromium relies on available storage to determine quotas, but fortunately there is already an active patch for that (one less job :). it would be interesting to retrieve this data automatically in other chromium forks as well, I do not know whether @arthuredelstein (for brave) or @SkewedZeppelin (for mulch) are interested. |
No changes made, just shifting the order of some patches
Please allow me to clarify your procedure and the results:
I guess it would be interesting to include ungoogled-chromium into your test-pool :) Is it doable? Do you need help with that? |
This might be due to the summing of the coordinates in the result. It's faster and makes integrity checks easier, but some entropy is lost. |
I was referring to the table for reporting data with markdown, it would have taken up too much space without giving any information
no, so far no special meaning. I only needed it to display a subset of data.
Yes
is just to make that set of values more readable (which would otherwise be in the format
yes.
yes, but the device could be the same, I mean, just the same, whereas I would also be interested in checking different devices of the same model.
yes and no, some would crash, others need the 32-bit version, mine is a first step, to rough out the problem.
if you mean the desktop version, no, it is not possible with browserstack since installation of new applications is not allowed for desktops.
find a free service similar to browserstack but for desktops that exposes different physical machines, but I don't think it exists. |
I don't think so, it would be too unlikely a coincidence, given the use of floating-point numbers. However, I came up with a way to understand it.
consider that my goal will be to define a screen size to be applied to all devices, for example a 412x1000, because obviously screen height is a fingerprint factor. |
Preparing the build of chromium vanilla (#1271)
I think, I'd convert your excel file into csv, prepare it and then send into Postgres for analysis via SQL.
Do you have the full result? I admit, hash collisions are very unlikely, but still there could be useful patterns. |
https://www.cromite.org/tests/creepjs.zip is the result prior to aggregation in excel.
no, no advantage in using it. |
I have not forgotten this, on the contrary. next step, create an action for this blessed test! |
For comparisons I also intend to attempt the build of brave with a modified version of chromium as a baseline (#772) |
Surprised? Why? |
Me? It is more like intrigued :) I'm looking forward to what interesting you may find. |
I have noticed that a lot, but really a lot, is done by adblock. i will also put adblock in vanilla chromium, to compare the differences between cromite and chromium. |
In terms of fingerprinting protection? |
well, if you want, you can consider it that way too. |
I am getting closer to the goal: as you can see, the rectangles are identical but the devices are different!
|
Would it be possible to detect your trick given the differing dpi? |
Well, the fact that the original values cannot be derived is part of the patch. Currently this is not the case. |
um... Actually, there is one thing I had not thought of, namely the elements docked at the bottom of the viewport. |
implementing this feature does not seem simple at all. I had to completely revise the first version I wrote: unfortunately it had problems during user pinch-to-zoom because setting a device-ratio to 1 grains the image during resize.
there is also this to verify https://web.dev/articles/device-pixel-content-box |
however, what seems strange to me, is that no one has ever investigated this and found a solution, but merely said that the differences in the production of the canvas used for fingerprinting are due to a different gpu, that's not true, that's not the case. |
there are three different types of coordinates in chromium to which three different types of zoom are linked:
note that cc is not really THE compositor, but rather a middleware between blink and viz, where the latter communicates to the gpu, I don't know whether for historical reasons or to have an additional layer to separate the logic between different platforms. |
I may have found a way by reusing the code provided for handling the rootScroller particularly https://chromium-review.googlesource.com/c/chromium/src/+/578293
Here again, I will try to reuse the code provided for disabling jit. |
bad luck haunts me... basically, with pinch-to-zoom active
the bug is due to the clamp (of offsets) made by cc between the viewport and the content area, which is wrong and causes the page to ‘jump’ up and down to the end of the scroll. |
Is Chromium susceptible to this bug? You could try filing a bug against Chromium and see if they could figure it out ;) |
yes, it is indeed a chromium behaviour, also visible in chrome.
I have opened other more serious bugs without getting an answer, so I am very sceptical, maybe I will try the mailing list of the blink team. |
I decided to start making this browser into something more serious.
To date, there is no test in cromite that indicates what bug and regressions there are in each release, and more importantly, what the differences are compared to chromium, even from the point of view of security if not privacy.
That is why I have always thought that automatic tests should be introduced.
What I need is an automatic test platform.
this is the idea:
this is a project within a project, so it is hard to imagine that it will not last for years.
The text was updated successfully, but these errors were encountered: