Meeting_2006-04-24

• Date: 24.04.2006
• Time: 09.30 Norw. time
• Place: Where we are
• Tools: SubEthaEdit, iChat

1. Opening, agenda review
2. Reviewing the task list from two weeks ago
3. Documentation - divvun.no
4. Corpus gathering
5. Corpus infrastructure
6. Infrastructure
7. Linguistics
8. name lexicon infrastructure
9. Spellers
10. Other issues
11. Summary, task lists
12. Closing

Opened at 10: 11.

Present: Børre, Thomas, Tomi, Trond, Saara

Absent: Maaren (sick leave), Sjur (paternal leave) Main secretary: Trond

Agenda accepted with additions under "Other".

Børre

• send out contracts with accompanying letter
  • Not done
• Gather public texts, preferrably also parallel ones
  • Done
• Continue converting text from input format to our xml
  • Not done
• convert nob and nno bible texts to be used as part of a parallel corpus
  • Not done
• review the paratext2xml converter
  • Not done
• convert smj NT to paratext
  • Not done
• Send out letters to the Iđut authors
  • Not done
• write docu for how to apply for a corpus user account (forms, recipients, etc)
  • Not done
• remove old corpus files from gt/sme/corp/ after Trond has cleaned it
  • Not done
• integrate generated corpus repository summaries in the Forrest site
  • Not done
• make free texts available
  • possibly e-mail Finnut about the resource (if Sjur goes on paternal leave)
    • check if this has been done
    • Not done
• fix bugs!

Maaren

• on sick leave throughout April

Saara

• Create a parallel corpora of the new testaments.
• add more texts to the graphical corpus interface
• grammatical searchability in the graphical corpus interface
• file a bug report of utf-8 check in xml-validation of the corpus files.
  • done
• make free texts available
  • also provide an xml-free version?
    • done
  • set up a weekly cron script (but only if there are new files)
    • not done.
• Discuss: allow for more than one file at a time when uploading a file.
  • done
• Implement links to parallel files in corpus header.
  • not done
• Turn on language recognition, skipping Finnish.
  • done
• add a possibility to upload whole documents for hyphenation (and also analysis, generation, etc)
  • not done, discussed in news
• add a log of every word/text uploaded/hyphenated/analyzed etc.
  • not done
• fix bugs!

Sjur

• on paternal leave

Thomas

• add all word boundaries
  • still working with sme-propernames
• work on compounding and derivation
  • not
• smj G3 issue
  • not
• sme G3 issue
  • not

Tomi

• move aspell UTF-8 suffix bug to Bugzilla
  • not done
• Document aspell infrastructure: finish doc/proof/spelling/X-spell/aspell.xml (it's almost done, but there are a couple of loose ends)
  • not done
• new proper name lexicon
  • implement data synchronisation of proper nouns between risten.no and CVS
    • not done
  • XQuery refactoring and code development for our proper noun editor
    • doing
  • new version of xml2lexc (based on ccat), should handle complex names correct: construct entries like we have now from the different parts of a complex name entry
    • not done
• read aligner docu, install, provide feedback
  • not done
• install and test Gobby, install new version of SEE
  • not done
• Set up the mechanism for the hash-mark transducer package
  • not done
• fix bugs!
  • not

Trond

• Unify anchor lists, and conflate nno and nob into nor
  • Done the unification, done letter A on nob/nno issue.
• remove deleted files from the CVS repository (in the Attic)
  • Done
• better smj NT text, get fin NT texts
  • Not followed up this one.
• Prepare a list of good candicates for first inclusion into the corpus.
  • Decided to follow the principle "biggest texts first".
• start and lead discussion and work on semantic features for disamb
  • Not done.
• Install Gobby
  • Done, was able to install 0.3. According to Sjur, 0.41 should be within reach for Mac users also, we should try to get 0.41 versions, since the two are not compatible with each other.
• get a key for Maaren in May
  • Not done
• install aligner, test it and give feedback
  • Not done
• correct hyphenation of word boundaries and exceptions
  • Worked quite a lot on this issue (as has Thomas), the file starts to be ok now.
• get/upgrade keys for Børre's room for Tomi and Thomas
• fix bugs!.

TODO:

• documentation on how to apply for a user account for the corpus repo ( Børre)
  • The item will be moved to the TODO list, again.

Trip to Sámi municipalities and ,,,,

Børre is going to a trip to Karasjok, Tana, Nesseby, Utsjoki and Inari, to visit the municipalities, parliaments and Sámi Council. In Karasjok he will try to visit Davvi Girji as well.

Collecting

See a previous meeting memo for what's to be done.

TODO: Send out the rest of the letters (Børre)

Odin

Sæth replied by e-mail, hasn't had time to follow-up, but will try to include us in their plans.

• Børre will weekly mirror the URL mentioned in Sæths e-mail, and add missing files to the corpus.

Olavi Korhonen's Lule Sámi dictionary.

• No news this week

TODO:   Børre to contact Olavi Korhonen and Kuhmunen

KIO Grafisk and the Iđut books

TODO:

• send letters to the authors (Børre)
• wait for the discussions with Davvi Girji
  • A talk with Brita Kåven, revealed that they would have a look at the contracts after easter.

Bible texts

We will get text from Finland, but still haven't received any. We have got the Swedish text from Sweden. As for the last html versions from Norway, Trond has not contacted them last week.

Swedish html has arrived, no paratext. Norsk bibelselskap has not sent corrected New Testament versions for sme, and not paratext for nno/nob.

TODO:

• convert smj NT to paratext (Børre)
  • Not done
• get fin, swe, nob and nno NT and OT in paratext format. (Trond)
  • No news

Min Áigi

Everything ok here.

TODO:

• Børre to contact Per Christian Biti on technical issues (how to transfer texts).

Kåfjord

Promised to send us texts, but nothing has arrived yet.

TODO:   Børre to contact them.

Sámi Instituhtta

Audhild Schanche has signed the contract. We will have to contact them about transferring the texts.

TODO:   Børre to contact them.

https: //giellalt.uit.no/lang/corp/corpus-summary.html

TODO:

Changes and updates because of the Divvun public tender

User account admin and infra: see previous memo.

TODO: see above under Documentation.

Automatic build of the content of our corpus repo: also see previous memo.

TODO:

• make free texts available
  • done weekly by a cron script (but only if there are new files) (Saara)
• Make a link, easily available, to these texts.

Free and non-free texts

More info in a previous meeting memo.

TODO:

• Check the status of the texts, again. (Børre, Trond)
• Rerun the conversion afterwards (Saara is the one with the magic spell)

Linking parallel files

DECISION: We'll keep the original filename, and store linking info in the header (has to be added manually).

TODO:

• develop the web interface for uploading to make it easier to add several documents in one go. (Saara)

More texts to the graphical corpus interface:

TODO:

• We would like to have more than the NT in the graphical interface http: //omilia.uio.no/CE/sami/ (Saara)
  • We add the largest texts first.
• We would like to have grammatical searchability, not only POS. (Saara, Trond)
• This presupposes a discussion with Oslo. (Trond and Saara to continue this discussion)
• For Lule Sámi: We would like to have a parallel corpus interface with NT (text only). This presupposes better quality texts (Trond, Børre)
  • Better Lule NT text still not made.
• The list of good candicates: The longest (admin) texts.
  • We need a ccat version of the script for analysing text, still keeping xml tags. (Tomi).

Top-two priorities:

1. Trond and Saara to discuss with Lars.
2. Lars to add text to the server.
3. Tomi to prepare for the parallel corpus.

Language recognition

TODO:

• turn on language recognition, skipping Finnish (Saara)
  • Done, it seems. Short word list Trond to make such lists.

some flag to write into the xsl file:

• method: do not run lg recognition
• method: Choose between these 2: nob, sme

TODO: Saara to implement this and to write a short documentation on how to write in the appropriate commands in the file-specifix xsl documents.

Aligner

Today, we have two anchor files in addition to the original one.

TODO:

• Read documentation and try out, give feedback to Bergen. (Trond, Saara, Tomi)
• conflate nno with nob into 'nor' (Trond)
  • Partly done, must go through and see.
• Saara to install the aligner, everyone to read the documentation
• Trond and Saara will continue this issue.

Hyphenator

Trond and Thomas have been updating the propernoun file with ^ tags. We need the tag in front of compound parts beginning in a vowel or in two or more consonants. Compound parts beginning with one consonant are handled correctly.

TODO:

• Remove the ^ signs from the UPPER level (just like the TV and IV tags are removed today. (Trond)
• Otherwise we wait with the following items until Sjur is back.
• correct hyphenation of word boundaries and exceptions (Sjur, Trond)
• add a possibility to upload whole documents for hyphenation (and also analysis, generation, etc) (Saara)
• we should log all and every word/text uploaded/hyphenated/analyzed etc
  • we'll do it, but it does not have first priority (Saara)

General - hyphenation

See discussion, open questions and decission in the previous meeting memo.

TODO:

• add all word boundaries (Thomas)
  • Done: All sme except propernoun-file. Trond has gone through the propernoun file and added ^ marks for the most common name elements, such as fjell, fjord, but there is still more to be done. Thomas has done propernoun-file A-N.
• Set up the mechanism for the hash transducer package. (Sjur, Tomi, Trond)
  • Not done, let us wait until Sjur is back.

TODO:

• Thomas and Trond to carry on marking ^ on the compound parts beginning in a vowel or in two or more consonants.

North Sámi

Semantic feature system

We postpone this issue until Linda has met with Eckhard in early May.

Lule Sámi

TODO:

• add the rest of the inc- words (Thomas)
  • everything added that is possible now, about 50 unknown words left+2 abbr. +moaddi etc (numerals)

TODO:

1. refactor and prepare risten.no for multiple collections:
   1. refactor the code into more and more specific components according to our folder hierarchy (Tomi, Sjur)
      1. things are moving forward
2. develop the needed XQueries and interface (Sjur, Tomi)
   1. developing
3. data synchronisation between risten.no and the cvs repo (Tomi)
   1. nothing this week
4. test and review when ready

Discussion postponed until Sjur is back.

Nothing until the new proper noun lexicon is in place. We don't have enough people to do both.

Discussion postponed until Sjur is back.

Bug fixing

50 open Divvun/Disamb bugs, and 25 risten.no bugs

Please help Saara with bug 279. Not much help...

Saara will contact Roy on this issue.

After the corpus issues have been somewhat settled, we should do a bug barnraising. ... and then a new one after the name lexicon is fixed.

Børre

• send out contracts with accompanying letter
• Contact Per Christian Biti (Min Áigi) on technical issues (how to transfer texts)
• Contact Kåfjord
• Gather public texts, preferrably also parallel ones
• Continue converting text from input format to our xml
• convert nob and nno bible texts to be used as part of a parallel corpus
• review the paratext2xml converter
• convert smj NT to paratext
• Send out letters to the Iđut authors
• write docu for how to apply for a corpus user account (forms, recipients, etc)
• remove old corpus files from gt/sme/corp/ after Trond has cleaned it
• contact Olavi Korhonen and Henrik Mikael Kuhmunen
• fix bugs!

Maaren

• on sick leave throughout April

Saara

• Create a parallel corpora of the new testaments.
• add more texts to the graphical corpus interface
• grammatical searchability in the graphical corpus interface
• make free texts available
  • set up a weekly cron script (but only if there are new files)
• Implement links to parallel files in corpus header.
• Implement turning off the language recognition in the xsl-file (and corpus.dtd).
• add a possibility to upload whole documents for hyphenation (and also analysis, generation, etc)
• add a log of every word/text uploaded/hyphenated/analyzed etc.
• fix bugs!

Sjur

• on paternal leave

Thomas

• add all word boundaries
• work on compounding and derivation
• smj G3 issue
• sme G3 issue

Tomi

• move aspell UTF-8 suffix bug to Bugzilla
• Document aspell infrastructure: finish doc/proof/spelling/X-spell/aspell.xml (it's almost done, but there are a couple of loose ends)
• new proper name lexicon
  • implement data synchronisation of proper nouns between risten.no and CVS
  • XQuery refactoring and code development for our proper noun editor
  • new version of xml2lexc (based on ccat), should handle complex names correct: construct entries like we have now from the different parts of a complex name entry
• read aligner docu, install, provide feedback
• install and test Gobby, install new version of SEE
• Set up the mechanism for the hash-mark transducer package
• fix bugs!

Trond

• Make nob into nor in the anchor list (A is done)
• better smj NT text, get fin NT texts
• Discuss Gobby 0.4 with Sjur
• get a key for Maaren in May
• install aligner, test it and give feedback
• correct hyphenation of word boundaries and exceptions with Thomas
• get/upgrade keys for Børre's room for Tomi and Thomas
• fix bugs!.

02.05.2006 09: 30

Sjur is on paternal leave.

Closed at 12: 04