baijianpeng
Karma: 3
|
Stupid idea of saving files in database by default - 2006/11/19 13:24
I was suprised to see there is such a very stupid idea in Remository: it will keep the files we uploaded at backend in the databased intead of in the folder we created for saving files, called /downloads or something similar.
Why you set it to save files in database by default ?
This will eat up database space quickly !!! Of course when users want to backup the database, the size become unconveniently big.
In our country, database space is always very small compared with the web space. So it is better to save fiels in a folder than in DB. A folder is also easy to manage than DB.
I can't imagin who had set such a idiot like setting. Do you have too large database and want to use it completely?
Hope you can correct this error.
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
vibez
Karma: -1
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 11:25
I fail to see what has made you so angry. remository offers the ability to use either the database or the filesystem.
There are several advantage of using the database compared to the file system. Martin has documented these advantages/disadvantges many time on this site.
Ok by default it is set to use the database, but seriously, why is it such a problem for you to review the configuration screen and make the appropriate changes you think would best serve your site.
There always has to be a default configuration and it will not suit everyone. So calm down and think before you post such nonsense next time.
|
|
|
| | Sorry, you do not currently have permission to write here. |
baijianpeng
Karma: 3
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 12:42
Yes, I was angry. Now I still think it should be set to off by default. Not all users are so familiar as we are. A default setting should reflect the habit of common users NOT experienced users.
There is also another thing very annoying. If I set the option to No at backend, then add a new file, I found that the new file will be saved in the database intead of the folder. That is to say, the setting is for Container, not for File. Once the container was set to be saved in DB, later setting of disabling it will not take effect on newer files in this same container. You should delete the container and add a new one to make later files be saved in a folder.
Now you see, if the user had uploaded 10 files before he realised that he had not set this setting to NO which we really wanted to, then he had only one option: delete the container and build a new one, then add all those 10 files manually again !
Shouldn't he be so angry ?
Yes, maybe saving files in DB has some advantage, but you should measure that if MOST users want to do this ? What is more, you also should better make this option for Files instead of for Container.
Do you agree ?
Post edited by: baijianpeng, at: 2006/11/20 07:43
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
vibez
Karma: -1
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 13:27
I agree it can be a little confusing when changing from database to filesystem storage. I'm not sure how it could be simplified though, maybe a little FAQ in the readme would help people understand the way in which remository handles files.
I think Martin decided to use the database as default as you get far less issues with file/folder permissions as well as better security.
Post edited by: vibez, at: 2006/11/20 08:35
|
|
|
| | Sorry, you do not currently have permission to write here. |
baijianpeng
Karma: 3
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 15:17
Yes, I believe that saving files in DB can offer high level of security guarantee. But not all users want so high security, many files to be downloaded are not so confidential. Maybe the anti-leech method is enough for these files.
For those special files, I think zipped and encrypted before uploading to the server to provide as downloading material is a good idea. You can send the decrypt code to user by e-mail or something more secured.
Now, I want to say another confused thing by this default setting.
I did not notice this until I uploaded two files to the Remository. Then I know I want to save files in folder. So I deleted the container and also deleted the entries in jos_downloads_files table. Next, I set the option to NO and built a new container, uploaded new files. Ok, they are in folder now.
However, my problem is, now the database seems too big. one of the tables named jos_downloads_blob is nearly 1MB size. But I don't dare to empty it since I don't know the structure. In fact, I have contact with Remository component less than one week.
So this is another disadvantage. If user disable the option, the component can't prune the tables automatically even after user deleted the whole container.
I think the authors should think it over before make new improvements. As I know, many users around me choose DocMan as their favorite component for downloading area. But in my opnion, Remository is better than DocMan.
Remository can be even better if they can improve it according to users' preferences, especially the very common users'.
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
vibez
Karma: -1
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 15:57
The jos_downloads_blob table is where remository stores the chunks of each file you have in your conatiners. So if you are no longer using the database to store your files, it is safe to empty this table.
The reason this table is not already empty id due to the way you deleted your files. You should of used the remository control panel to delete these files. That way the blob table would of been pruned automatically and jos_downloads_files table updated correctly.
If you are using 3.40. There would of been an even quicker way. For example, I have over 4gb of files stored in my database. I decided that I wanted to move them all into the file system.
No problem, just change the "store in database" to 'no', then add an absolute path to each of your new containers. Remository then copies all your files out of the database into the specified folder automagically.
So as you can see, even if you don't like the default settings, changing them only takes a few clicks
|
|
|
| | Sorry, you do not currently have permission to write here. |
admin
Karma: 98
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 17:54
Thanks for coming to my aid vibez 
As you said, the relative merits of database or file system have been discussed at length here. The only thing I will add here is what I consider a particularly important point:
The default is database (and is likely to stay that way) for the very simple reason that it should ALWAYS work in any hosting environment, without any special knowledge whatsoever on the part of the user. In fact, I keep trying to find ways to make Remository work immediately on installation - 3.40 now inserts a menu item and creates a sample container to make it as easy as possible.
Using the file system is a harder option and often requires some understanding of paths and file permissions (hardly anyone really does understand them properly) and is a cause of innumerable support questions from people who cannot get Remository to work.
For many users, who have quite a small repository of files, the database will remain a perfectly good option and easy to manage. But as vibez said, moving a container to the file system could hardly be made any easier.
[Incidentally, within reasonable bounds, there is no very good reason to prefer using disk space outside the database rather than allowing the database to grow].
Post edited by: admin, at: 2006/11/20 12:55
Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com |
|
|
| | Sorry, you do not currently have permission to write here. |
baijianpeng
Karma: 3
|
Re:Stupid idea of saving files in database by defa - 2006/11/20 23:31
Thanks to vibez for detailed help and patience. I also agree that the default setting will allow users start easily as well as the automatically built menu item.
But this doesn't mean the uesr heimself determined to use the DB than disk space. He just did not expect such a strange setting by default. You all think saving files in DB is better than in a folder, that is because you don't know situation outside your home.
In China, hosting providers always limit the DB size to a very small number such as 50MB in common while the web space was given as 250MB. Of course, this is for personal website and small enterprise website if they don't have too much images and videos. Such a poor space is often sufficient to build a small site.
Now you can see, the DB size is considered very rare here. More over, when users want to backup the DB, your option will make it difficult because the memory limit on server is set to 36MB by default in China. When they want to restore a big SQL file due to Remository's default setting they will meet extreme difficulty since the memory limit will break the restoration. I have heard of too much complaining of taking pains when restoring a big DB file.
Maybe you will say China is not the whole world. But you should know we have the biggest group of internet users which may have surpassed 1000 million. Joomla users also grow very fast here. We ALL used to store files in folder. If you want to change the comtom by such a default setting, you will see users go to DocMan instead.
Vibez said if I change the setting to NO then Remository v3.4 will move all those files out the DB. But unfortunately I did not see this happened. In fact, I thought it should work this way. You can see in my first post that I DID changed the setting to NO and expected to see the files would be moved automatically. But nothing happened. This was the very point made me so angry.
At last, I solved this problem by deleting the container and building a new one. Thanks God I don't have too many files uploaded before I noticed this setting.
If you are sure the changing of option will auto-move the files, OK, you needn't change anything else. But I really doubt it.
Also If you only want experinced users to accept your slogan instead of average people like it, we do not need to discuss any more here.
At least one thing is sure, after I translated this component I shoul warn our people to change this option immediately after installation. Of course, on my website where they will download the translated version. Don't worry, I will keep the author's name untouched.
Post edited by: baijianpeng, at: 2006/11/20 18:38
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
admin
Karma: 98
|
Re:Stupid idea of saving files in database by defa - 2006/11/21 14:57
Well, I really think you're being pretty unreasonable, and that you would make more progress by a less aggressive approach
It is stretching a point to say that I don't know the situation outside my "home". I don't actually claim to have a comprehensive knowledge of hosting practices anywhere at all, but what experience I do have extends to Europe, North America and Australasia. And, with due respect, since I certainly don't know enough to generalise about UK hosting (even though that is where I live), I'm reluctant to take one person's word for the situation throughout the entire Chinese hosting scene. I listen to all the views expressed here, and then make the best judgements I can in development. Sorry if you don't like the results.
Certainly within my experience, there are hosts that place severe restrictions on database size, but plenty of other (no more expensive hosts) who do not care how space is used, simply placing an overall limit. Moreover, you have ignored my point that quite a few Remository users simply need to offer a modest number of quite small files - e.g. documents or PDF files.
The overall default setting is shown at the top of the Remository control panel, so should be seen every time a user reviews the operation of Remository. It means exactly what it says. It defines the "Default storage for NEW containers". It has no effect on existing containers, or on files.
However, what vibez actually referred to is quite correct. A container that is using database storage (and this is shown in the list provided by "show containers" so ought to be clear enough) will have a filepath that is blank. To change the database to using the file system, simply set a legitimate absolute path. When the container is stored, the files will be moved. There was no need for you to delete a container. Files can be moved into the database by the reverse process, or moved to a different part of the file system by modifying the absolute path in the container. It seems to me that the user has total control and is never in a bind that cannot be resolved easily. I really don't see any grounds for your anger.
Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com |
|
|
| | Sorry, you do not currently have permission to write here. |
baijianpeng
Karma: 3
|
Re:Stupid idea of saving files in database by defa - 2006/11/21 23:16
Yes, from your opinion, my anger seems unreasonable and my operation a little stupid. But you don't realised that you are an very skilled "user", while I am speaking on behalf of very common users. Especially those in China among which some can't read English well.
I believe you know well about many countries. But you don't know one fact: the internet users in China may equals the sum of other countries.
You let me remember one thing: the Catch 22. You know how to make a great component, but don't know how to market it. Another fact you don't know is, users in Chna like DocMan more than Remository.
Vebiz is right. There should be put a correct absolute path for the container if changed the option. I had forgotten to do this so I did not see the files been moved.
But don't you think adding a path is more complicated than setting NO as default? If set to NO, people want to store files in DB can change it to YES without adding any path. This operation is easier than vice versa.
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
admin
Karma: 98
|
Re:Stupid idea of saving files in database by defa - 2006/11/22 16:37
I've never said anything about your operation being stupid. (Although, as I've said, I don't understand the rationality of restricting particular kinds of disk space, rather than the total usage, and I am not clear why large database tables should create problems on restore - that's not at all the same as suggesting that anything is stupid).
But I don't think you're listening to me at all. The choice of database as default is made PRECISELY to make life easier for "very common users". It is a response to the numerous difficulties they have encountered with file systems. As I said, I listen to all the comments made. My own expertise is actually quite patchy, but in any case, Remository is absolutely NOT pitched to any kind of skilled user. It isn't pitched at all, it simply develops in response to pressures from user feedback.
I really don't know what is the point of your statements about the number of Chinese internet users. Nor do I mind if some people choose to use Docman - they are welcome - I even have it as a download here! People should use what they find best. Why should I object to that?
Should I be "marketing" Remository? Why? What benefit would that bring to anybody? I do my best for Remository users, but if their number increases all that happens for me is that I have more unpaid work to do! Should I be making a special effort to bring that about?
There is no need for anyone to enter a path at all. If you really do not want to use the database, then change the default, delete the sample container and create another one (or more). BUT you will have to tangle with paths if Remository has not managed to find a good path for the default, or if you want to make your installation at all secure. I can't help that, it just comes with the file system!
Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com |
|
|
| | Sorry, you do not currently have permission to write here. |
baijianpeng
Karma: 3
|
Re:Stupid idea of saving files in database by defa - 2006/11/22 18:40
Yes, I also hate such a small disk space and DB sapce here. Unfortunately it is the fact. Most small sites in China can't and won't afford a bigger but more expensive hosting server because it IS expensive than in other countries. I told you the number of internet users in order to remind you that most of the future Joomla suers will come from China not those from dozens of other countries.
Well, I want to stop arguing on this topic. Anyway, it is an option, not a fixed feature. I was angry partly because I was not told to add a path after changed the option to NO.
I like remository because it offered such a more secure option but I don't like it to be the default one though I can handle it without problem myself.
You can still insist on your opinion, I don't expect you will hear opinions similar to mine since people having the similar situation as mine are using DocMan.
Forget my words about marketing. I didn't mean money but maybe I chose a wrong word.
Let me give a summary, I will still use remository, I will publish Chinese translation for it and warn our people to notice this option. You are right in your situation and so do I.
Ok, we stop argue on this since we can't persuade each other.
Thank you for your effort on this component.
http://www.joomlagate.com
Chinese Joomla! Users' Portal |
|
|
| | Sorry, you do not currently have permission to write here. |
vibez
Karma: -1
|
Re:Stupid idea of saving files in database by defa - 2006/11/22 18:50
Group Hug 
I'm sitting on the wall here. I can see both side of the discussion. The fact is that every user has different requirements. The default options will be great for one user, is not so good for the other.
At the end of the day the software is free and has the option to be configured to suit both camps.
|
|
|
| | Sorry, you do not currently have permission to write here. |
admin
Karma: 98
|
Re:Stupid idea of saving files in database by defa - 2006/11/24 09:43
I'm sorry, but this doesn't seem to me to have been a constructive discussion. Nothing has been dispassionately explored and no progress made. If baijanpeng feels strongly about it, given that Remository is open source, there is absolutely nothing preventing him from creating and distributing his own modified version with defaults set to whatever he thinks appropriate.
Thanks for your kind thought, vibez
Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com |
|
|
| | Sorry, you do not currently have permission to write here. |
claudia23
Karma: 0
|
Re:Stupid idea of saving files in database by defa - 2007/05/08 20:53
I have found your discussion quite useful, mainly bacause I finally understand how this part of Remository works.
But, I have the following problems:
1) To give some kind of security to my documents
2) The system manager that obliges me to remove documents from DB because he cannot backup the DB (jos_downloads_blog gives error)
Then I have to move files to filesystem, but unfortunatelly they are not encrypted.
I noticed when I uploaded a file that if I had the DB option to yes, Remository changed the file name, while if I have this option to no, the file name remains untouched.
could be possible to change automatically the file name even if the files are stored in the file system?
I know this is not a real solution but this could be something.
Thanks a lot
Claudia
|
|
|
| | Sorry, you do not currently have permission to write here. |
admin
Karma: 98
|
Re:Stupid idea of saving files in database by defa - 2007/05/09 17:27
I'd have preferred you to have started a new thread, all the same, I don't much like the title of this one
The motivation for storing files in the DB is not primarily security. The main reason is to create a working system regardless of the file permission issues that plague some types of hosting and baffle novice users. Database storage is much more consistent in its behaviour across different hosts.
If you'd like to explain what you want security against it would be easier to comment! If files are stored in the file system, they can be made totally secure against direct download by one or both of:
1) Place the files outside the document root
2) Use .htaccess to deny all access to the files - you don't need Apache to be able to serve them for Remository to work.
Not sure what you mean by file names being changed - the name of the submitted file is always preserved and used for downloads, regardless of the mode of storage. Remository 3.42 inserts the file ID into the name on disk (and removes it again on download) so as to allow duplicate file names to coexist.
The old anti-leach system used to prefix file names, but this makes management very difficult as you can't easily list the files in a sensible order. It should not be necessary given the alternative mechanisms available.
There is a new FAQ on file storage.
Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com |
|
|
| | Sorry, you do not currently have permission to write here. |
Lazi
Karma: 0
|
Re:Where files are? (in Download folder) - 2008/12/22 01:20
Hello!
Maybe someone will give me an advice/answer. Even if I change savings files to file system and I add new container and put some file. I can't see it (this file) on server in download folder. I install already Remository two times and still is the same problem. My version of Remository is 3.43 and Joomla 1.5.8
Thanks in advance
Lazi
EDIT:
Next day I try do something with fresh mind and everything is ok. Reason of my problem was that I put all the time files from remote localization and I thinking that they are should appear in Download folder of Remository ))
Merry Christmas and Happy New Year 
Post edited by: Lazi, at: 2008/12/22 08:59
Post edited by: Lazi, at: 2008/12/22 09:00
|
|
|
| | Sorry, you do not currently have permission to write here. |
|
Forum
