Nasty filter bug

Brian Milby brian at milby7.com
Wed Jan 24 16:37:02 EST 2024 

I just want to clarify that this isn’t the regex version of filter but the wildcard pattern version.  It is much less complicated than regex.  Square brackets are used to group characters to be matched so you can use [abc]* to match any item that starts with a, b, or c.  The dictionary entry for this is pretty good.

Brian Milby
brian at milby7.com

> On Jan 24, 2024, at 3:35 PM, Craig Newman via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Brian.
> 
> The original post tried to filter a string by filtering (without) that actual string, and was interested in why that did not yield empty. Intuitively, the result of such a filter operation ought always to be empty. The presence of the char “[“ is the “culprit”. That is as far as I took it.
> 
> Anyway, you state that the char “[“ is a regex special character, which explains a lot. The other two, “?” and “*” filter without issue, though, and I guess one has to be a regex user to see why.
> 
> I am not.
> 
> Craig
> 
>> On Jan 24, 2024, at 2:53 PM, Brian Milby via use-livecode <use-livecode at lists.runrev.com> wrote:
>> 
>> Your test misses the actual issue:
>> 
>> on mouseup
>>  local tStr
>>  local tFilter
>>  put "a*b" into tFilter
>>  put "anything bold" into tStr
>>  filter tStr with tFilter
>>  put tStr
>> end mouseup
>> 
>> Will yield "anything bold"
>> 
>> while using the following 2 lines:
>>  put "a?b" into tFilter
>>  put "a*b" into tStr
>> will yield "a*b"
>> 
>> Finally, using the following 2 lines:
>>  put "a[?]b" into tFilter
>>  put "a?b" into tStr
>> will  yield "a?b"
>> 
>> The point is that you can end up with issues if you are counting on "*" and
>> "?" to filter literally.  They don't - they are wildcards and can match
>> anything (multiple char or single char respectively).  The "[" is also
>> special.
>> 
>> To ensure that your filters work properly if you do not want to use any
>> wildcards (i.e. match * and ? literally) would require you to change *, ?,
>> and [ to [*], [?], and [[].
>> 
>>> On Wed, Jan 24, 2024 at 1:19 PM Craig Newman via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>> 
>>> OK, instead of working I did this:
>>> 
>>> on mouseUp
>>> 
>>> repeat with y = 1 to 255
>>> 
>>> put "XX" & numToChar(y) & "XX"  into temp
>>> 
>>> filter temp without temp
>>> 
>>> if temp <> "" then put y & return after accum
>>> 
>>> end repeat
>>> 
>>> answer accum
>>> 
>>> end mouseUp
>>> 
>>> There are two characters that prevent the filter command from doing its
>>> job: ASCII 91 (“[“) and ASCII 10, the return char.
>>> 
>>> 
>>> 
>>> Craig
>>> 
>>> 
>>>> On Jan 24, 2024, at 12:51 PM, Craig Newman via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>> 
>>>> Brian.
>>>> 
>>>> Nope. Those two chars pass through the filter, er, filtered.
>>>> 
>>>> Again, I did not test the entire character set.
>>>> 
>>>> Craig
>>>> 
>>>>> On Jan 24, 2024, at 11:05 AM, Brian Milby via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>>> 
>>>>> The only other two that would cause issues are ? and * which are single
>>> and multiple char wildcards respectively.
>>>>> 
>>>>> Brian Milby
>>>>> brian at milby7.com
>>>>> 
>>>>>> On Jan 24, 2024, at 10:21 AM, Craig Newman via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>>>> 
>>>>>> I did not test the ASCII set exhaustively, but the culprit is the
>>> char “[“ (ASCII 91). Any other char (including “]”) in the string works
>>> correctly, that is, nothing is left after the filter command executes.
>>>>>> 
>>>>>> I do not know enough to say whether that particular char  does
>>> something to the filter command, which may use regex somehow in its inner
>>> workings.
>>>>>> 
>>>>>> Craig
>>>>>> 
>>>>>> Craig
>>>>>> 
>>>>>>> On Jan 23, 2024, at 9:45 PM, Brian Milby via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>>>>> 
>>>>>>> Not sure this is really a bug.  The default is to match a
>>> wildcardPattern.  If you want to match [ then you must use [[] in the
>>> pattern.
>>>>>>> 
>>>>>>> Brian Milby
>>>>>>> brian at milby7.com
>>>>>>> 
>>>>>>>>> On Jan 23, 2024, at 9:02 PM, Neville Smythe via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>>>>>> 
>>>>>>>> Try this in the msg box:
>>>>>>>> 
>>>>>>>> put "aaa[bbb" into tStr; put line 1 of tStr into tLine; filter tStr
>>> without tLine; put tStr
>>>>>>>> I get (using MacOS, LC 9.6.11)
>>>>>>>> 
>>>>>>>> aaa[bbb
>>>>>>>> 
>>>>>>>> That is to say, the line is not filtered out.
>>>>>>>> 
>>>>>>>> And:
>>>>>>>> 
>>>>>>>> put "aaa[bbb" into tStr; filter tStr with tStr; put tStr
>>>>>>>> 
>>>>>>>> produces an empty string instead of the original string.
>>>>>>>> 
>>>>>>>> The bug occurs if the line contains the character “[“ anywhere; any
>>> lines containing that character are ignored by both filter with and filter
>>> without.
>>>>>>>> 
>>>>>>>> This is really serious, because I rely on the filter command a lot,
>>> as I would think do many other developers!
>>>>>>>> 
>>>>>>>> Don’t know if it occurs with other characters, but I have never seen
>>> it before. I discovered it when filtering lines with regular expressions.
>>> Other special regexp characters I have tested do not trigger the bug.
>>>>>>>> 
>>>>>>>> Neville Smythe
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> use-livecode mailing list
>>>>>>>> use-livecode at lists.runrev.com
>>>>>>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>>>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> use-livecode mailing list
>>>>>>> use-livecode at lists.runrev.com
>>>>>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> use-livecode mailing list
>>>>>> use-livecode at lists.runrev.com
>>>>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>>> 
>>>>> _______________________________________________
>>>>> use-livecode mailing list
>>>>> use-livecode at lists.runrev.com
>>>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>> 
>>>> 
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

More information about the use-livecode mailing list