Home arrow Forum

Remository Forum

 


Boardwalk :: Forum List Remository Support Remository 3.4+
mtilink

Karma: 0  
Add Structure Problem with non-latin files - 2010/07/29 13:49 Hello,

I tried using this feature "Add Files Already on Server" on a folder filled with folders and files in Arabic.

Everything works fine and the files are uploaded with the right structure of files, at least this is my guess. When browsing the folder where the files should move (the main container folder), I can see the folders/files in the right structure with their full and right Arabic names.

It's just that the files/containers have no names in remository when viewing them in the frontpage or admin control panel.

I tried uploading files one by one and they show up correctly so remository has no problem with uploading Arabic filenames and keeping the same filename when downloading. It might be just an "add structure" issue when dealing with different character sets.

By the way, my LANG parameter in /etc/sysconfig/i18n is en_US.UTF-8 and I changed it to ar_DZ.utf8 and tried the "Add Structure" again with the same result.

Any suggestion is highly appreciated.
Thanks
  | | Sorry, you do not currently have permission to write here.
admin

Karma: 98  
Re:Add Structure Problem with non-latin files - 2010/07/30 11:24 I think that the problem here is a discrepancy between the character encoding used in the file system and UTF-8 used by the CMS and Remository.

The file system may well be using UCS2, but I'm not sure that there is a simple way to discover the character set that is used by the file system. Different operating systems and file systems are liable to work in different ways in this respect, and clear documentation is hard to come by.

PHP will see the file paths simply as byte strings, and so long as the names only use ASCII, they will look the same as if they were in UTF-8. But if a language is in use for file names that is substantially different from ASCII, the file system encoding may well be incompatible with UTF-8. I suspect that is why you are seeing a problem. Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com
  | | Sorry, you do not currently have permission to write here.
mtilink

Karma: 0  
Re:Add Structure Problem with non-latin files - 2010/07/30 22:57 Thanks a lot for taking the time to send your feedback.

I read about character encoding in file system, apache and php. Finally, I knew that the problem is not in them.
While I wanted to resort to viewing the files I have online by putting them directly in folder viewed by apache users. I noticed that files/folders' names were encoded using windows-1256 (Arabic) and not UTF8.
I used the following command to change files/folders' names encoding from cp1256 (arabic) to UTF8 inside the folder "results":
Code:

 convmv -f cp1256 -t utf8 -r --notest ./results


Please refer to this page http://www.linux.com/archive/feed/58689 for more information.

I tried the "Add Files Already on Server" again and it worked fine.
  | | Sorry, you do not currently have permission to write here.
admin

Karma: 98  
Re:Add Structure Problem with non-latin files - 2010/07/31 12:17 That is excellent! And thanks for providing the information on how you solved the problem. Martin Brampton aka Counterpoint
http://aliro.org
http://black-sheep-research.com
  | | Sorry, you do not currently have permission to write here.
Boardwalk :: Forum List Remository Support Remository 3.4+

Aliroboard Forum Component 1.2
Aliro Software

Menu

Advice Notes

Login

Subscribe to Premium Support

Get priority support for Remository and Glossary, sign up now for a Premium Support monthly subscription:

Your Remository user name

Or purchase a year's support:

Your Remository user name

Recommended SEF

SEF Advance

Who is Online

Remository welcomes guests and visitors

We have 8 guest online