Ganjoor Development Blog

Until recently, for complex editing I relied on an HTML editor which after sending new content regenerated the whole verses and information for a poem. This would result in loss of data which was somehow tolerable in my opinion.

After recent changes in poems structures (adding sections), using this editor would result in loss of more important data which could not be ignored.

So, as the first step, I disabled the old editor for poems and now I am trying to implement editing some metadata and also adding the ability to deleting and adding verses to the user poem editor. So, it would work as a suggestion collector and edit would take place in a two phase process (suggest and moderate phases).

People can suggest corrections for poems and a moderator (currently me, myself) reviews these corrections and approved or rejects them.

Applying these corrections relies on regenerating poem data which potentially is not safe.

I now have created some complex structures in order to support poem blocks, and regenerating poem data might result in loosing some of these structures.

So now I need to make editing poems process more reliable. At the first place I relied on the old methods because the new poem correction method does not support some operations like adding or deleting couplets or verses and also restructuring a poem format. These features are complex ones and I need to make my civilized editor support them in order to be able to get rid of the old wild and savage one.

I have recently (around two months ago) moved ganjoor.net to a new server and used FileZilla Server to transfer files (I don’t recall on the source or destination machine) and recently found out that some of mp3 file (poem recitations) on the new server are corrupted. I am not 100% sure about the cause of corruption but FileZilla server is the most probable reason.

Fortunately I have backups for these files but I cannot replace the whole files with the backed up ones because people might have uploaded replacements for their old recitations and replacing them with the old files causes other issues.

I maintain an MD5 checksum for MP3 files in the accompanying XML files which contains metadata and synchronization information for the MP3.

So I am going to use the combination of my backup files and the MD5 checksum for the files to resolve this issue.

There always been an issue for labeling poems prosody meters and rhymes within ganjoor.net, because these data are attached to the whole poem which although for majority of them suits well, there are parts of ganjoor.net which are not compatible with this approach.

Specifically for normal (non-poetic) works like Gulistan which in each part might include multiple blocks of different poems with different meters and rhymes this issue is visible, and apart from requiring the mentioned metadata to be attached to different sections of a page (post/poem), it needs a way to make these different parts of the text distinguishable from each other.

Although BANDs in a multi-band poems are visually distinguishable, their rhymes need to be stored separately in order to find similar poems based on combination of their whole prosody meters and rhymes.

There are other geometrically complex models which need different virtual sections inside them to make them ready for finding similar poems:

Each couplet is a Masnavi has its own independent rhyme in order to find similar poems.

Although ganjoor.net does not have new poem formats having different blocks of a poem separate and distinguished is a requirement for these types of poems.

This are the types of issues I am trying to resolve here.

After improving TOC generation, I developed a utility to assist me in migrating old TOCs to the new ones.

During examination of the results and trying to improve non-alphabetic tables of contents I encountered some old issues with the data including misplaced poems, repetitive items and things like that.

The repetitive data comes from the fact that there were works of a poet which were available only in a selection when I uploaded them and then, their full or more complete work became available. The old selection during this time had got some additional metadata including user comments and some data would be removed if I was about to delete them. They also might become linked from various pages on the internet and deleting them caused broken links.

Now I have added a feature which allows me to keep a deleted page URL in a new target metadata, and fix the broken link issue by redirecting to the replacement.

Some basic CRUD utilities, including deleting poems and importing poems from SQLite to a specific category were missing. I resolved them and after that I intend to create a utility which would find similar (repetitive) poems and move their metadata to their replacements and then delete/redirect them to their final replacements.

I’ve previously added some new fields to some database models to be able to achieve this.

First of all some poet pages HTML contents are not all generated based on database models and had been manually edited.

Resolving this issue helps me to be able to be sure their contents are fully generated and does not need editing manually.

I also want to improve the user interface for these kinds of pages (poet pages and category pages) during this task.

PS. It is done. I’ve changes these pages from this UI:

To this one:

I have to break this poem in two due to the issue reported by a user:

This is the user comment on this:

I have an already implemented solution for a simpler version of this problem which allows me to break the LAST poem of a category in two:

I had prepared this feature for breaking long poems (some parts of شاهنامه) in two or more shorter sections.

Breaking the last poem is easy, because you create a new poem and you are not concerned on performing complex movements among lots of poems (moving poems corresponding comments, narrations, images and …).

But breaking a poem which has some other poems after it in its corresponding section (category) is a more complex problem because I need to perform lots of modifications to the poems located after this new poem.

This is this issue which I’m trying to resolve here.

This is the Ganjoor Development Blog maintained by me.

Its main purpose is to keep a minimal technical documentation log for things I (or may be later we) do to improve ganjoor. My native language is Persian and even though it might be more reasonable to have this blog written in Persian I prefer to use English in order to improve my English language writing skills.

Ganjoor gitub projects are the main subjects of this blog.