Home » Public Forums » FoxTrot Search User Forum » Support for searching full path?
| Support for searching full path? [message #1343] |
Mon, 07 February 2022 23:58  |
Atlas
Messages: 140
Registered: August 2009
|
Senior Member |
|
|
The single biggest reason why I keep renewing my Foxtrot subscription, is because it supports searching through file contents AND metadata at the same time -- such as the parent folder name. So if the content contains "foo" and the parent folder's name contains "bar", then the Foxtrot search "foo bar" will show the file, even though "foo" and "bar" do not appear together in the content or parent folder name, because Foxtrot is able to search for the contents and all metadata like the parent folder name together. This is a really powerful feature, because I can effectively label my files by putting all related files under one folder, and now all files in that folder will "inherit" that folder's name as a metadata. Without this ability, I would have to do a two stage query, which is not ideal.
However, I just realized that Foxtrot doesn't actually index the full path name; and it only indexes the parent folder name. So if I have a file that contains "foo", and I put it inside a subject folder "bar", and I put that under a topic folder "bazz"; then the Foxtrot search "foo bar bazz" will not give me any result, because foxtrot it will only see that the file content contains "foo", and the parent folder name contains "bar", but it won't see the "bazz" in the full path name.
Is it a big ask to request that Foxtrot indexes the full file path in the metadata, instead of just the parent folder name? Or maybe the capability is already there but I'm not using Foxtrot options correctly? Please let me know.
Thanks for your consideration.
[Updated on: Tue, 08 February 2022 00:00] Report message to a moderator |
|
|
|
| Re: Support for searching full path? [message #1344 is a reply to message #1343] |
Tue, 08 February 2022 09:17  |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
That is true; the full path is not indexed, and thus searching for [bazz] won't find files inside subfolders of bazz.
However, you can use advanced filters for this. For example:
[contents, any metadata or filename] [includes all of the words] [foo bar]
[then apply advanced filter] [full path] [contains the string] [ignore case] [bazz]
[contents only] [includes all of the words] [foo]
[other metadata] [includes all of the words] [bar]
[then apply advanced filter] [full path] [contains the string] [ignore case] [bazz]
[contents only] [includes all of the words] [foo]
[then apply advanced filter] [full path] [contains the regular expression] [case insensitive] [/bazz/[^/]*/[^/]*$]
Jérôme - FoxTrot Engineering
[Updated on: Tue, 08 February 2022 10:31] Report message to a moderator |
|
|
|
| Re: Support for searching full path? [message #1351 is a reply to message #1344] |
Fri, 18 February 2022 13:47 |
Atlas
Messages: 140
Registered: August 2009
|
Senior Member |
|
|
Background
Thanks, Jerome, for giving a thoughtful response on this question. Previously, I have discussed this exact issue with Pierre Bernard from HoudahSpot, and I also had to take some time to explain why indexing the fullpath gives additional querying power that existing methods -- like what you described -- do not. Again, I thought that Foxtrot indexes the fullpath (not just the parent path), and it's the reason why I thought it's the most powerful search tool on the market for Mac. Please hear me out on why indexing full path gives users extraordinary querying power.
Scenario 1 Use Case
The methods you've describe will work IF I know where the term "foo", "bar", and "bazz" occurs. In particular, it would work if I know that "bazz" is somewhere in the full path; in which case, Foxtrot will allow me to search the full path with the term I'm looking for. However, the problem is when I DON'T know whether "foo", "bar", and "bazz" occur in the file content, file name, or the path; and what I'm searching for is all documents that have at least one occurrence of these terms anywhere in the content or full path metadata. For example, a document may have (1) "foo" and "bazz" in the content and "bar" in the path, or (2) "foo" in the content and "bar" and "bazz" in the path. Using the existing methods that you described, if I apply filter to the full path to only search for documents that contain "bazz" in the path, then it would find the second type of document but it would fail to find the first type of document. Again, the existing methods will only work if I already know whether a keyword occur in the full path. If I only have a list of two keywords, then I would only have to run two separate query to check whether the first keyword occur in the full path or the second keyword occur in the full path, but if I have a list of 4 or 5 words (that are potentially combined in a boolean query), then clearly you can see it's not a trivial task trying to search for a large list of keywords that can occur anywhere in the content, metadata, or fullpath. It WOULD be simple, of course, if the fullpath is indexed just like the parent folder name is currently indexed. In such case, someone like me can solve the whole problem with just ONE foxtrot query.
Now, why would someone want to search for a list of words that can occur anywhere in the content or full path metadata? Because users don't always remember the structure of how they've added descriptions to their files. For example, I might structure my folders as (1) "virtual environment" > "python" > "file about vagrant" or (2) "virtual machine" > "python environment" > "file about vagrant". Some months or years later I might want query all documents related to "python environment vagrant". This query would work if I use the second folder structure (because the parent folder name and file content include all the keywords), but it would fail if I used the first folder structure. The problem is users don't always remember how they've structured their folders and which folder level they've added the keyword descriptions that they now remember.
Therefore, indexing the fullpath (and not just the parent folder) is extremely powerful because users don't have to remember how their database is structured, which makes the search tool much more flexible and powerful.
Scenario 2 Use Case
You may say "why don't you just have a very flat folder structure so that the parent folder contains all the necessary keywords" or "why don't you make sure that the keywords in one folder name is contained in the folder name of all its children folders". Well, I've tried to do that before. However, there is at least one scenario where users are restricted in how they can add keywords to folders, and indexing the fullpath makes a lot more sense. The common example is whenever you downloaded a project, and that project already has a particular folder structure. For example, a project might be organized as "Research project on inflation drivers by Author Foo" > "Data Bar", "Report Bazz", and this folder structure makes sense because the parent folder tells me the subject and author of the research, and the children folder tells me the different components of the research project. Since the author is likely to only put their name on the report, but not on the data or the models they used to generate on the report, using the search string "Author Foo Data Bar" will give me no result. Another common example is when I download materials for a course, and it is organized into folders such as "Notes", "Slides", and "Lectures". So if I downloaded a some course materials by "Author Foo", and the notes about "Subject Bar" are in the folder "Notes", then searching for "Author Foo Subject Bar" will not give me any result, unless content of the notes also contain the name of the author (which doesn't always happen). Granted, this scenario is not as bad as the first scenario above, because I would usually realize "Oh, the data set probably don't have the author name", and so I will use the methods you've described and search for the author's name in the fullpath. However, it is intuitive and convenient if the query "Author Foo Subject Bar" gives the result that my mind would expect.
Thus, sometimes people are restricted to using deep folder structures that have a lot of children, or it's not sensible to rename the existing folder structure. In such case, indexing the fullpath and not just the parent folder is very useful..
Final Thoughts
1. I really thought that Foxtrot indexes the fullpath. Was this a feature that was available in the past and was removed?
2. Foxtrot's ability to index the parent folder name is a clear differentiator for me compared to other search tools like HoudahSpot, Devonthink, etc. I hope that Foxtrot can build upon this existing capability and add indexing of the fullpath as well, and I hope that the reasons I've articulated provides some good justifications.
Thank you for your considerations.
[Updated on: Fri, 18 February 2022 13:47] Report message to a moderator |
|
|
|
| Re: Support for searching full path? [message #1363 is a reply to message #1351] |
Sun, 20 February 2022 18:33 |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
1. No, FoxTrot never indexed the full path
2. I don't think indexing the full path is useful in most circumstances; and I think it would have serious drawbacks. Especially in regard to the home folder: in a typical situation, all indexed files are inside the home folder; and the home folder name is the name of the user. So, searching for the user name would find all files…
There will be an experimental and unsupported option in version 7.5 to index the relative path instead of the name of the parent folder (the relative path is the full path, stripped from the indexed folder path). To enable it, you will have to paste this command in Terminal.app:
defaults write com.ctmdev.FoxTrotShared UseRelativePathForParentFolder -bool YES
Jérôme - FoxTrot Engineering
|
|
|
|
| Re: Support for searching full path? [message #1366 is a reply to message #1363] |
Wed, 23 February 2022 15:34 |
Atlas
Messages: 140
Registered: August 2009
|
Senior Member |
|
|
|
Thanks for the consideration. I think indexing the relative path is a GREAT solution to the problem; not hung up on indexing the home folder, because I would never search for it. I look forward to testing out the relative path indexing feature in 7.5. Again, this is the biggest killer feature of Foxtrot for me, so thank you!
|
|
|
|
| Re: Support for searching full path? [message #1381 is a reply to message #1343] |
Fri, 18 March 2022 19:02 |
Bruce Lewis
Messages: 4
Registered: July 2020
|
Junior Member |
|
|
If the full path is not indexed, FoxTrot would not know if I have moved a file. Therefore, the index will always be out of date after a move. Is that correct.
For instance, I just searched and found dozens of files with the message "This file (or one of its parent folders) has been moved, etc.
I thought that that would be corrected when the Index updated and was surprised when it didn't.
How should I deal with files or folders that are moved within the existing folder structure?
|
|
|
|
| Re: Support for searching full path? [message #1382 is a reply to message #1381] |
Fri, 18 March 2022 19:48 |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
Quote:If the full path is not indexed, FoxTrot would not know if I have moved a file. Therefore, the index will always be out of date after a move. Is that correct?
Correct
Quote:I thought that that would be corrected when the Index updated and was surprised when it didn't.
It should. FoxTrot has no knowledge of what you did (move, rename, delete, restore...). If you moved or renamed a file, it will consider that the original file has been deleted, an a new one created.
One exception is when one of your indexed folders (those you added to the "indexed locations" pane) has been moved or renamed: FoxTrot considers that it is "temporarily unavailable" (e.g. unmounted external drive, or disconnected network location), and thus it does not remove its contents from the index when you update it. You have to explicitly remove this folder from the indexed locations to do so (and add the renamed / moved folder, if applicable).
Jérôme - FoxTrot Engineering
|
|
|
|
| Re: Support for searching full path? [message #1397 is a reply to message #1382] |
Tue, 05 April 2022 06:51 |
Atlas
Messages: 140
Registered: August 2009
|
Senior Member |
|
|
|
You mentioned in Feb that there will be an option in version 7.5 to index the relative path. Is this feature included in the recent 7.5 beta release? I look forward to this change.
|
|
|
|
| Re: Support for searching full path? [message #1398 is a reply to message #1397] |
Tue, 05 April 2022 09:28 |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
Yes; our FoxTrot hidden preferences page has been updated:
The name of the folder containing a file is normally indexed as part of the “other metadata”, allowing to find files directly in this folder by searching the folder name. If you want to index the relative path instead (full path of the file, stripped from the path of the indexed folder), so you can find all files at any depth inside a folder (this requires rebuilding the index):
defaults write com.ctmdev.FoxTrotShared UseRelativePathForParentFolder -bool YES
Jérôme - FoxTrot Engineering
|
|
|
|
| Re: Support for searching full path? [message #1401 is a reply to message #1398] |
Wed, 06 April 2022 01:58 |
Atlas
Messages: 140
Registered: August 2009
|
Senior Member |
|
|
|
Thanks for continuing to support this feature. Foxtrot's ability to search the path together with file contents in one search string via Foxtrot Query is the biggest reason I still recommend this tool to others. The downside is the bugs when using Foxtrot Query. I still have a backlog of bugs that I haven't posted yet, because Foxtrot seems busy and I want to slowly feed the bugs with highest priority first so they can get focus first. I'll file more bug reports in the coming weeks now that 7.5b came out.
|
|
|
|
|
|
Goto Forum:
Current Time: Thu Dec 19 15:42:34 GMT+1 2024
|
] 
FoxTrot Search Forum
Search
Help
Members
Register
Login
Home
