About Mantis performance and scalability

General discussion of Mantis.

Moderators: Developer, Contributor

Post Reply
Uno-Di-Noi
Posts: 8
Joined: 16 May 2020, 17:58

About Mantis performance and scalability

Post by Uno-Di-Noi »

Hello all.

I've inherited a Mantis installation at the company I work for, where it has been used as a Ticketing Management System since about 5 years. Our installation of the MantisBT software works fine and has a very low maintenance burden, which is great.

This Mantis installation (v1.2.19) I've found that has been heavily customized, both in the PHP source code and also with extensions to the MySQL database tables so as to be able to handle per-project Tags, which is a critical feature for us, so I don't see that I will be able to update to a more modern Mantis version any time soon. But I am not worried about that, because our current Mantis setup works just fine for our needs ("if it's working, don't fix it").

However, I am worried about the long term performance and scalability of our Mantis installation. We currently have around 15,000 issues in Mantis, and most of them have at least one attached file (so we have about 12,000 attachments in the disk directory assigned for that). The attached files are small (usually screenshots and excel files), so the space used by them in the web server disk is not a concern for me, however the high number (several thousands) of files in disk is beginning to worry me.

We expect to keep using our Mantis installation at least for ten more years, and my projection is that in that time frame we will reach about 50,000 issues in Mantis. And here is where 50,000 files in the same disk directory looks like a bit much to me.

So my questions are: Do I have to be worried about that?, Is there any MantisBT-supported strategy to place the attached files in disk into subfolders by year or similar approach?

---

Also, although I don't expect our MantisBT installation to live that long, how will MantisBT perform if it reaches 200,000 issues?

And finally, how does MantisBT deal with the end of the UNIX epoch and the dates it stores in the database?
atrol
Site Admin
Posts: 8366
Joined: 26 Mar 2008, 21:37
Location: Germany

Re: About Mantis performance and scalability

Post by atrol »

The maximum number of files per directory and also the number where you get performance issues depends on the file system you use
https://stackoverflow.com/questions/466 ... directory/

If you have more than one project in Mantis, you can configure one directory per project.

The number of issues should not affect that much performance, at least as long as you don't do full text search for all issues.

Concerning UNIX epoch, ask again in 2037 and be prepared to upgrade your Mantis installation :wink:
Please use Search before posting and read the Manual
Uno-Di-Noi
Posts: 8
Joined: 16 May 2020, 17:58

Re: About Mantis performance and scalability

Post by Uno-Di-Noi »

atrol wrote: 16 May 2020, 19:59 The maximum number of files per directory and also the number where you get performance issues depends on the file system you use
https://stackoverflow.com/questions/466 ... directory/

If you have more than one project in Mantis, you can configure one directory per project.

The number of issues should not affect that much performance, at least as long as you don't do full text search for all issues.
Thanks for your reply.

That Stackoverflow page presents useful information, although there are contradictory reports in there about my case: the MantisBT's web server I've been tasked with taking care of is a 32bit intallation of CentOS 5.11 (very old, I know, but it works and it's not accessible from the Internet) with EXT3 filesystems. Some people on that page say EXT3 has a limit of about 60,000 files per directory, while others say they have 8 million files in a single ext3 directory. So I guess I'll have to do some tests myself. I'll probably make a clone of the web server as a testing virtual machine and then I will do some stress tests in that VM. (Speaking of which, I probably should convert that server to a VM anyway so I don't have to worry about it's old hardware dying some day; it runs on enterprise iron but spare parts will become harder to find as time passes.)

Yes, we do have several projects in our Mantis installation, and each one is set up with its own directory to save attachments. But only one of those projects is heavy in attachment usage, and that is the one project that worries me regarding the number of files in a single directory, as time goes by and the number of files grow it may reach 50,000 files in a single directory.

Is there any strategy built-in into MantisBT to save the attached files for a single project to disk into subfolders by year of by the first letter of the filename? In case there is none, is that problem in the radar of the development team as a feature request?, and what are the plans about it?
atrol wrote: 16 May 2020, 19:59Concerning UNIX epoch, ask again in 2037 and be prepared to upgrade your Mantis installation :wink:
So, I understand that modern versions of MantisBT have no problem with the UNIX epoch, right? Or perhaps this problem is just an issue of the underlying operating system and database server and therefore it's transparent to Mantis?
atrol
Site Admin
Posts: 8366
Joined: 26 Mar 2008, 21:37
Location: Germany

Re: About Mantis performance and scalability

Post by atrol »

Uno-Di-Noi wrote: 16 May 2020, 20:49 Is there any strategy built-in into MantisBT to save the attached files for a single project to disk into subfolders by year of by the first letter of the filename? In case there is none, is that problem in the radar of the development team as a feature request?
There is no built-in strategy and there is no activity / plan around that topic. Related feature request https://mantisbt.org/bugs/view.php?id=7186
Uno-Di-Noi wrote: 16 May 2020, 20:49 So, I understand that modern versions of MantisBT have no problem with the UNIX epoch, right?
No, I meant:
Who knows what will be the current state of operating systems, PHP, databases and MantisBT in January 2038.
E.g. who knows what this function will deliver in 2038 on a 64 bit system https://www.php.net/manual/en/function.time.php
who knows if Mantis will still use the function in later versions, who knows ...

Your current installation will certainly fail, so you have to start doing something not later than 2037.
Please use Search before posting and read the Manual
Uno-Di-Noi
Posts: 8
Joined: 16 May 2020, 17:58

Re: About Mantis performance and scalability

Post by Uno-Di-Noi »

Thank you @atrol for your reply.

You have been most helpful, and I appreciate that.

I will keep going, hopefully for 10 more years, with our current Mantis installation, and then we will see what the future brings. I am confident our current Mantis installation will be able to endure 50,000 issues and 50,000 in-disk attachments. After those 10 years, God will provide. :wink:

Also, thank you very much for this great software and for making it open source. It beats ANY and ALL competing solutions in this niche (Ticketing Management System) by a wide margin!
TheNetMasterAlex
Posts: 1
Joined: 12 Mar 2024, 01:32

Re: About Mantis performance and scalability

Post by TheNetMasterAlex »

Hello;

based on what I've read, your main concern seems to be about the scalability of the directory or disk for a storage model over a 10-year period, particularly relating to the number of files and the limitations of the partition or format.

I believe the approach you're taking does not align with your expectations.

If I had your concerns, I would first use a Long Term Service operating system to ensure validity and stability over 10 years (a classic case for On Premise servers).

Secondly, to bypass file limits, as mentioned here, I would use a directory per project, and for these directories, you can mount disk units that could be real, virtual, SAN, or NAS; this way, you create a highly scalable framework with the required agility.

If you want to go further and enhance this scalability beyond 10 years to preserve the server as legacy and avoid hardware degradation issues, use a virtual machine that's easy to clone and migrate within an easily manageable ecosystem.

That is, something you can move easily.

For this point, the suggestion of volumes mounted on directories will always be virtual.

In essence, it's a complex solution for a complex concern.
Post Reply