Home » Public Forums » FoxTrot Search User Forum » Help us test FoxTrot Search 7's performance increases
| Help us test FoxTrot Search 7's performance increases [message #946] |
Thu, 28 November 2019 21:58  |
CTM info
Messages: 179
Registered: September 2009
|
Senior Member |
|
|
Dear FoxTrot-Search customers,
Following many months of rewriting critical performance bottlenecks, we are pleased to release for testing to our group members FoxTrot version 7.0b2.
If you want to help us out, kindly do the following before upgrading to 7.0b2:
We initially only ask that you rebuild (not just update) one of your most commonly used index files with the existing version 6.6.3, and send us the indexer log file (which you can access from the Manage Indices window in Foxtrot Pro)
Ideally, this would be an index of a somewhat substantial size, one which would take 30 minutes or more to rebuild.
Please send the resulting logs NOT to this list, but instead to our support mail address by opening a Support case from the Help menu.
In the subject, after the case number, type the word PERFORMANCE, and we will track this afterwards.
Once this is done, you may proceed to download the FoxTrot Search 7.0 beta 2 seeds, depending on your interest, either:
FoxTrot Professional Search: http://www.ctmdev.com/beta/foxtrot_professional.dmg
FoxTrot Search Server: http://www.ctmdev.com/beta/foxtrot_search_server.dmg
FoxTrot Personal Search: http://www.ctmdev.com/beta/foxtrot_personal.dmg
To contribute a report (strongly encouraged !), please use the Help: Report menu, or if you find the issue worthy of other’s attention, use the discussion group. The first report should be a version 7 rebuild of the same index as above.
THANK YOU !
The FoxTrot-search team
at CTM Development SA
PS: As a motivational hint, depending on configurations, we have seen indexing performance increases ranging between 180% and 700% in version 7 !
---
FoxTrot Pro 7.0b2 release notes
▾ Performance improvements
• Much improved indexing time on multicore machines, especially when the index is stored on an SSD. This is however highly dependent on the type and size of the indexed files, and on the hardware configuration.
• 7.0b2 makes a smarter use of available processor cores when extracting indexable text.
▾ New search features
• [Pro] Added an “All items of type” criterion to find all indexed files of a given file type (or even all indexed files, regardless of their file type), without searching for any particular text string. This can also be used in conjunction with the filters, to perform advanced filename, date, or regular expression searches.
• Added a “filename extension” categorizer that can be used in addition to “item kind”.
• Added a visible / invisible categorizer.
▾ Other searching or indexing improvements
• Added more options to decide whether to index or not files in packages, applications and frameworks.
▾ User interface improvements
• Dark mode support (macOS 10.14 Mojave and later).
• [Pro] The proximity slider now shows a more precise tooltip.
• [Pro] When multiple search criterion fields are visible in the search window, empty ones are now always ignored (you previously had to remove unused ones).
• [Server] Added contextual menu to reset login info (username and password) for a given server, in FoxTrot Admin.
• Added a “Search…” menu item, with command-L shortcut, to move the keyboard focus to the search field.
• The contextual menu “reveal in finder” now shows the complete path as a submenu.
▾ Miscellaneous improvements
• Use a modern API for language identification on macOS 10.13 and later.
• Highlighting found occurrences did not work for documents created with recent versions of Pages, Numbers and Keynote.
• Added a hidden preference to use the NaturalLanguage framework for language identification on macOS 10.14 and later (defaults write com.ctmdev.FoxTrot UseNaturalLanguageIdentification -bool YES). This is slower, but may be more accurate.
• [Pro / Server] More secure network encryption (TLS 1.3) for shared indices.
• Estimation of the remaining indexing time is now more reliable.
• Some files could previously be added to the blacklist, although they had made the extractor crash only once; some third-party extractors can crash randomly, even when processing valid files.
• Indexer log files now contain much less useless information.
▾ Bug fixes
• The indexer could become unresponsive, and could refuse to stop, when compacting a very large index stored on a hard drive.
▾ System requirements
• macOS 10.12 Sierra or later.
--
---
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #947 is a reply to message #946] |
Fri, 29 November 2019 03:18  |
Xiangping Liu
Messages: 4
Registered: November 2019
|
Junior Member |
|
|
Thanks for the invitation. But I have been busy with my work. I will spare
some time late to help you. Your product is amazing. Hope you will do
better.
Xiangping Liu
School of Geography
Beijing Normal University
No. 19 Xin Jie Kou Wai Street, Haidian District,
Beijing, 100875, P.R. China
CTM Development - FoxTrot Search 于2019年11月29日周五
上午4:58写道：
> Dear FoxTrot-Search customers,
>
> Following many months of rewriting critical performance bottlenecks, we
> are pleased to release for testing to our group members FoxTrot version
> 7.0b2.
> If you want to help us out, kindly do the following *before* upgrading to
> 7.0b2:
>
> We initially only ask that you *rebuild * (not just update) one of your
> most commonly used index files with the existing version 6.6.3, and send us
> the indexer log file (which you can access from the Manage Indices window
> in Foxtrot Pro)
>
> Ideally, this would be an index of a somewhat substantial size, one which
> would take 30 minutes or more to rebuild.
>
> Please send the resulting logs NOT to this list, but instead to our
> support mail address by opening a Support case from the Help menu.
>
> In the subject, after the case number, type the word PERFORMANCE, and we
> will track this afterwards.
>
> Once this is done, you may proceed to download the FoxTrot Search 7.0 beta
> 2 seeds, depending on your interest, either:
>
> *FoxTrot Professional Search*:
> http://www.ctmdev.com/beta/foxtrot_professional.dmg
>
> *FoxTrot Search Server*:
> http://www.ctmdev.com/beta/foxtrot_search_server.dmg
>
> *FoxTrot Personal Search*: http://www.ctmdev.com/beta/foxtrot_personal.dmg
>
> To contribute a report (strongly encouraged !), please use the Help:
> Report menu, or if you find the issue worthy of other’s attention, use the
> discussion group. The first report should be a version 7 rebuild of the
> same index as above.
>
> THANK YOU !
>
> The FoxTrot-search team
> at CTM Development SA
>
> PS: As a motivational hint, depending on configurations, we have seen
> indexing performance increases ranging between 180% and 700% in version 7 !
>
> ---
>
> *FoxTrot Pro 7.0b2 release notes*
>
> ▾* Performance improvements*
> • Much improved indexing time on multicore machines, especially when the
> index is stored on an SSD. This is however highly dependent on the type and
> size of the indexed files, and on the hardware configuration.
> • 7.0b2 makes a smarter use of available processor cores when extracting
> indexable text.
>
> ▾* New search features*
> • [Pro] Added an “All items of type” criterion to find all indexed files
> of a given file type (or even all indexed files, regardless of their file
> type), without searching for any particular text string. This can also be
> used in conjunction with the filters, to perform advanced filename, date,
> or regular expression searches.
> • Added a “filename extension” categorizer that can be used in addition
> to “item kind”.
> • Added a visible / invisible categorizer.
> ▾* Other searching or indexing improvements*
> • Added more options to decide whether to index or not files in packages,
> applications and frameworks.
> ▾* User interface improvements*
> • Dark mode support (macOS 10.14 Mojave and later).
> • [Pro] The proximity slider now shows a more precise tooltip.
> • [Pro] When multiple search criterion fields are visible in the search
> window, empty ones are now always ignored (you previously had to remove
> unused ones).
> • [Server] Added contextual menu to reset login info (username and
> password) for a given server, in FoxTrot Admin.
> • Added a “Search…” menu item, with command-L shortcut, to move the
> keyboard focus to the search field.
> • The contextual menu “reveal in finder” now shows the complete path as a
> submenu.
> ▾* Miscellaneous improvements*
> • Use a modern API for language identification on macOS 10.13 and later.
> • Highlighting found occurrences did not work for documents created with
> recent versions of Pages, Numbers and Keynote.
> • Added a hidden preference to use the NaturalLanguage framework for
> language identification on macOS 10.14 and later (defaults write
> com.ctmdev.FoxTrot UseNaturalLanguageIdentification -bool YES). This is
> slower, but may be more accurate.
> • [Pro / Server] More secure network encryption (TLS 1.3) for shared
> indices.
> • Estimation of the remaining indexing time is now more reliable.
> • Some files could previously be added to the blacklist, although they
> had made the extractor crash only once; some third-party extractors can
> crash randomly, even when processing valid files.
> • Indexer log files now contain much less useless information.
> ▾* Bug fixes*
> • The indexer could become unresponsive, and could refuse to stop, when
> compacting a very large index stored on a hard drive.
> ▾* System requirements*
> • macOS 10.12 Sierra or later.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "foxtrot-search" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foxtrot-search+unsubscribe«~at~»googlegroups«|dot|»com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/foxtrot-search/3FC8DCAD-2A 02-4DD9-8645-DD09E756C455%40ctmdev.com
>
> .
>
--
---
|
|
|
|
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #954 is a reply to message #953] |
Wed, 11 December 2019 14:35 |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
> Darren Ingram wrote:
>
> Yes, it made a difference on a very large index I have (from over a week to just under a day). If this is the "big bang" for v7 I'd love to see a nicer display of data for v8.
Please feel free to elaborate on this: what kind of “nicer display” would you love to see?
Thanks for your feedback
Jérôme - Foxtrot Engineering
--
---
Jérôme - FoxTrot Engineering
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #955 is a reply to message #953] |
Wed, 11 December 2019 16:23 |
FoxTrot Engineering
Messages: 406
Registered: April 2020
|
Senior Member |
|
|
> Darren Ingram wrote:
>
> Yes, it made a difference on a very large index I have (from over a week to just under a day).
Did you manage to store this very large index on an SSD? Large index files (i.e. about 10 GB or more) are quite slow when stored on a hard drive; another option would be to create multiple indices instead of a large one, each being smaller than 10 GB, and build them one at a time.
Jérôme - Foxtrot Engineering
Jérôme - FoxTrot Engineering
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #957 is a reply to message #955] |
Thu, 12 December 2019 12:11 |
Darren Ingram
Messages: 12
Registered: May 2018
|
Junior Member |
|
|
I tried once but "something" eat up the remaining free disk space. I
suspected (without evidence) that FT used it up to update the indices (a
sort of temporary file) and I have since not had time to try again and
isolate it as a problem or not.
It is not possible to create smaller indices due to the underlying software
used to create the source documents.
I will try and clear some space in my schedule to have another try.
Darren
On Wed, 11 Dec 2019 at 17:23, Foxtrot Engineering wrote:
>> Darren Ingram wrote:
>>
>> Yes, it made a difference on a very large index I have (from over a week
> to just under a day).
>
> Did you manage to store this very large index on an SSD? Large index files
> (i.e. about 10 GB or more) are quite slow when stored on a hard drive;
> another option would be to create multiple indices instead of a large one,
> each being smaller than 10 GB, and build them one at a time.
>
> Jérôme - Foxtrot Engineering
>
>
> --
>
> ---
>
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #958 is a reply to message #946] |
Thu, 12 December 2019 17:38 |
jonathanalix via foxt
Messages: 51
Registered: May 2019
|
Member |
|
|
FWIW I have 12 folders on an external SSD catalogued. They total 1.9TB for
225,600 items.
The indexes are stored on the internal HD, which is a fusion drive (in the
Library folder).
The indexes are 50GB in size, the biggest is 21GB and has indexed 111,500
items.
(I haven't yet tried rebuilding that one with the new beta).
Are you suggesting I would get better/faster results if all indexes were
under 10GB?
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #959 is a reply to message #958] |
Thu, 12 December 2019 18:27 |
CTM info
Messages: 179
Registered: September 2009
|
Senior Member |
|
|
Hello,
The very first thing I would suggest is that you store your indices not on a fusion drive but on an SSD. That should make a difference.
And yes, remaining under 10GB per index, while not an exact value, will indeed produce better performance than 21GB.
Regards,
jean michel/ctm qa
> On 12 Dec 2019, at 17:38, 'Ajk Sanders' via foxtrot-search wrote:
>
> ?
> FWIW I have 12 folders on an external SSD catalogued. They total 1.9TB for 225,600 items.
> The indexes are stored on the internal HD, which is a fusion drive (in the Library folder).
> The indexes are 50GB in size, the biggest is 21GB and has indexed 111,500 items.
> (I haven't yet tried rebuilding that one with the new beta).
>
> Are you suggesting I would get better/faster results if all indexes were under 10GB?
>
>
>
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #960 is a reply to message #959] |
Tue, 17 December 2019 17:35 |
jonathanalix via foxt
Messages: 51
Registered: May 2019
|
Member |
|
|
Hi
I will keep the indices on the startup (Fusion) drive for now because I
need to be able to search them when the SSD is not plugged in.
I have pared down the folders and now have more indices but smaller ones.
Thanks
On Thursday, 12 December 2019 17:28:27 UTC, foxtrot-search wrote:
>
> Hello,
>
> The very first thing I would suggest is that you store your indices not on
> a fusion drive but on an SSD. That should make a difference.
>
> And yes, remaining under 10GB per index, while not an exact value, will
> indeed produce better performance than 21GB.
>
> Regards,
>
> jean michel/ctm qa
>
> On 12 Dec 2019, at 17:38, 'Ajk Sanders' via foxtrot-search foxtrot...@googlegroups.com > wrote:
>
> ?
> FWIW I have 12 folders on an external SSD catalogued. They total 1.9TB for
> 225,600 items.
> The indexes are stored on the internal HD, which is a fusion drive (in the
> Library folder).
> The indexes are 50GB in size, the biggest is 21GB and has indexed 111,500
> items.
> (I haven't yet tried rebuilding that one with the new beta).
>
> Are you suggesting I would get better/faster results if all indexes were
> under 10GB?
>
>
>
>
>
|
|
|
|
| Re: Help us test FoxTrot Search 7's performance increases [message #961 is a reply to message #960] |
Tue, 17 December 2019 20:10 |
CTM info
Messages: 179
Registered: September 2009
|
Senior Member |
|
|
Hello
If I may make a recommendation, it is that at the first time you build or rebuild indices you do so on the SSD. That will make the induction process much faster, then you can copy them over to your Fusion drive. Don’t forget to close the index before copying and re-open it from the new location after the copy is done.
regards
jean michel / ctm qa
> On Dec 17, 2019, at 5:35 PM, 'Ajk Sanders' via foxtrot-search wrote:
>
>
> Hi
>
> I will keep the indices on the startup (Fusion) drive for now because I need to be able to search them when the SSD is not plugged in.
>
> I have pared down the folders and now have more indices but smaller ones.
>
> Thanks
>
>
>> On Thursday, 12 December 2019 17:28:27 UTC, foxtrot-search wrote:
>> Hello,
>>
>> The very first thing I would suggest is that you store your indices not on a fusion drive but on an SSD. That should make a difference.
>>
>> And yes, remaining under 10GB per index, while not an exact value, will indeed produce better performance than 21GB.
>>
>> Regards,
>>
>> jean michel/ctm qa
>>
>>>> On 12 Dec 2019, at 17:38, 'Ajk Sanders' via foxtrot-search wrote:
>>>>
>>>
>>> FWIW I have 12 folders on an external SSD catalogued. They total 1.9TB for 225,600 items.
>>> The indexes are stored on the internal HD, which is a fusion drive (in the Library folder).
>>> The indexes are 50GB in size, the biggest is 21GB and has indexed 111,500 items.
>>> (I haven't yet tried rebuilding that one with the new beta).
>>>
>>> Are you suggesting I would get better/faster results if all indexes were under 10GB?
>>>
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google Groups "foxtrot-search" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to foxtrot...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/foxtrot-search/d478c3d4-e9 67-49f7-aff8-785d0f4f44e3%40googlegroups.com.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "foxtrot-search" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foxtrot-search+unsubscribe«~at~»googlegroups«|dot|»com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/foxtrot-search/ab9c7b56-18 60-452c-9405-d8b2068337ad%40googlegroups.com.
--
---
|
|
|
|
Goto Forum:
Current Time: Sun Dec 22 07:21:31 GMT+1 2024
|
] 
FoxTrot Search Forum
Search
Help
Members
Register
Login
Home
