windows defender issues? & other AV issues?

[Richard Gaskin]

J. Landman Gay wrote:
 > I'm willing to bet that any slowdown in the SE not related to AV
 > intervention is probably the new auto-complete features. I've
 > turned most of them off and I see no speed decrease.

Same here.  But since the LC IDE is written in LCS, anything it does to 
trigger this widely-documented issue with Win Defender will sooner or 
later show itself in at least some of our apps.

They key question is: which language features are most affected?


A Test
-------
Given the nature of AV, it seemed file I/O would be a likely candidate, 
but to pin that down for sure I dug up a test script that touches a wide 
range of LCS features, running it with both Defender's Real-Time 
Protection (RTP) on, and then off.

The script was the last of a series of general performance assessment 
scripts started by Malte back in 2014, during the early v8 series.  Many 
of the elements identified through the various contributions in that 
forum thread helped point the way to dramatic increases in 
frequently-used operations like lineoffset, file I/O, and others.  There 
are many good test suites out there, including some from Mark Talluto, 
Curry, and others.  Many may have similar breadth; I chose this one 
because I'm familiar with the specifics of its scope:
https://forums.livecode.com/viewtopic.php?f=67&t=22072&p=135950#p135752

I modified the script only slightly so that it runs in the GUI LC rather 
than LiveCode Server, and with both RTP on and off.


Findings
--------
Here are the results:

RTP ON
System Version: Win32 NT 10.0
--
Test_BuildList: 187 ms
Test_LineOffset: 3741 ms
Test_LineAccessByNumber: 642 ms
Test_LineAccessForEach: 36 ms
Test_ArraySplit: 95 ms
Test_EncodeArray: 90 ms
Test_DecodeArray: 114 ms
Test_ArrayAccess: 60 ms
Test_Merge: 1409 ms
Test_BuildFilePath: 505 ms
Test_FileTextWrite: 6869 ms
Test_FileTextRead: 192 ms
Test_FileBinWrite: 9017 ms
Test_FileBinRead: 74 ms


RTP OFF
System Version: Win32 NT 10.0
--
Test_BuildList: 192 ms
Test_LineOffset: 4012 ms
Test_LineAccessByNumber: 719 ms
Test_LineAccessForEach: 39 ms
Test_ArraySplit: 99 ms
Test_EncodeArray: 113 ms
Test_DecodeArray: 143 ms
Test_ArrayAccess: 76 ms
Test_Merge: 1609 ms
Test_BuildFilePath: 579 ms
Test_FileTextWrite: 515 ms
Test_FileTextRead: 186 ms
Test_FileBinWrite: 376 ms
Test_FileBinRead: 74 ms


Most tests are close enough that any differences can be accounted for 
with expectable fluctuations in general system performance due to the 
effects of background tasks.

But writing files - damn! Now I understand what the reporters in the 
forums have been talking about.

The magnitude of the speed hit makes me wonder exactly what RTP is doing 
- detailed analysis of the file I/O buffer?

Whatever it is, it's worth thinking about in terms of how it will affect 
our own apps.


Implications for Our Own Apps
-----------------------------
The LC executable is signed, so while that may mitigate other AV issues, 
apparently that has no effect with the impact of file I/O ops on RTP.

The auto-complete feature you mentioned here seems very relevant for all 
of us, because IIRC they use SQLite for that, where the driver's need to 
jump around the b-tree structure can trigger a fair bit of disk I/O.

Even with those off, I would imagine that if you have the Dictionary tab 
open in the SE you'd see a similar impact, since Dict lookups are made 
as the selection changes.

If there's any good news here, it's that there doesn't appear to be 
anything the team can do in the engine to avoid triggering RTP's impact.

But the bad news may impact many of us:  With apparently all disk writes 
affected, any use of a persistent local store will take a hit.

This includes not only flat files, but databases like SQLite.

Reading appears to have minimal impact, but while that won't affect flat 
files, some DBs may modify bits of some DB files when performing even 
read operations, so we can't completely rule out impact on what we think 
of as read-only DB operations.


Dealing With RTP
----------------
As a paranoid security freak, I can't in good conscience recommend to 
end-users to turn of any feature of their security kit.

But on balance, it's worth noting that other AV packages do not seem to 
impact storage performance this horribly, if at all.  And many of the 
more popular alternatives are generally considered better than Windows 
Defender.

So while it seems a bit heavy-handed to tell customers which AV package 
to use, Microsoft's unusually aggressive approach to monitoring writes 
is impactful, and from the many stories with other apps we can find 
around the web, sometimes seriously.

Maybe this is one more reason to move everything to the cloud, to end 
the tyrrany of overzealously monitored local disk I/O. :)

I honestly don't have the quick-fix answer that will keep our customers 
happy with disk-intensive apps.

Sugggestions?

-- 
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com