Meeting_2007-11-12

• Date: 12.11.2007
• Time: 09.30 Norw. time
• Place: Internet
• Tools: SubEthaEdit, iChat/Skype

1. Opening, agenda review
2. Reviewing the task list from last week
3. Documentation - divvun.no
4. Corpus gathering
5. Corpus infrastructure
6. Infrastructure
7. Linguistics
8. name lexicon infrastructure
9. Spellers
10. Other issues
11. Summary, task lists
12. Closing

Opened at 09: 44.

Present: Børre, Ilona, Risten, Sjur, Thomas, Tomi, Trond

Absent: Per-Eric

Agenda accepted as is.

Børre

• move Steinar's error markup in the xml files to (a copy of) the original
  • not done
• follow-up support for the sami languages in OpenOffice.org
  • not done
• fix Unicode bug in Hunspell conversion java code
  • haven't found any solution
• fix bug 550
  • not done
• install Leopard on the G5 and the Xserve
  • Done
• move giellatekno.uit.no (including Bugzilla) and www.divvun.no to the G5.
  • bugzilla is not available yet, haven't been able to compile the necessary perl module for database access.
• add paradigm testing
  • not done
• fix Windows CD installation bug
  • not done
• fix bugs!

Ilona

• lexicalise smj missing words
  • In letter s
• add smj proper nouns
  • done, but without me.
• other smj tasks, ask Thomas
  • Not asked.

Maaren

• lexicalise actio compounds

Per-Eric

• lexicalise words from the Olavi missing list
• fix bugs!

Risten

• finish the design/text for the CD and the cover
  • not finish
• set up CD-printing printer
  • Leif Åge has ordered a new printer
• try to burn a CD at SD
  • done

Saara

• add new XSL/XML headers for proofing test docs
• Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not)
• add nested error markup to xml conversion
• add hunspell testing support
  • done

Sjur

• work on the XML name editor/risten.no integration
  • nope
• set up risten.no on the G5 again
  • still no work
• test new and nested error markup
  • not yet
• get command line hyphenator for automated testing of the hyph-lexicons
  • recieved a first version, it contains the hyphenation bug, but it lets us set up the hyphenation test bench
• add hyphenation testing
  • not yet
• add hunspell testing
  • done
• add paradigm testing
  • almost done
• fix bug 550
  • no
• fix hyphenation of derivations, inflections
  • done
• follow-up support for the sami languages in OpenOffice.org
  • done - it looks promising at the moment, but the OOo deadline for getting into OOo 2.4 is this week - otherwise we have to wait until version 3: (
• fix Windows CD installation bug
  • not yet
• fix bugs!

Thomas

• sme->smj lexicon conversion to build bilingual lexicon resources
  • not this week
• add smj proper nouns
  • added
• check for bad hyphenation
  • not this week
• look at test cases still not behaving properly
  • worked a lot
• fix hyphenation of derivations, inflections
  • done
• sme->smj name conversion
  • done
• fix bugs!
  • worked

Tomi

• Hunspell lexicon conversion
  • not done
• sme->smj lexicon conversion to build bilingual lexicon resources
  • not done
• fix Unicode bug in Hunspell conversion java code
  • done
• fix hyphenation of derivations, inflections
  • done
• fix bugs!
  • fixing

Trond

• sme->smj lexicon conversion to build bilingual lexicon resources
  • Progress, progress
• fix hyphenation of derivations, inflections
  • Meeting held, the linguistic side seems ok, but awaiting speller testing to see whether we are there.
• sme->smj name conversion
  • Done.
• telephone meeting with Sjur and the faroese group re faroese speller
  • Still not been able to get hold of the two of them at the same time.
• fix bugs!.
  • Nothing done here.

Nothing new.

TODO:

• fix bug 550 ( Børre, Sjur)

Nothing.

http://giellatekno.uit.no and http://www.divvun.no are now hosted on the G5. Bugzilla is not yet up and running again.

ssh www.divvun.no = G5
ssh giellatekno.uit.no = G5
ssh 129.242.220.111 = xserve

Jabber support = iChat server:

• encrypted chats
• permanent chat rooms
• stored and forwarded messages
• archive of chats, indexed and searchable

Addresses of the format:

sjur@divvun.no
trond@giellatekno.uit.no

TODO:

• add Jabber account in iChat (all)

North Sámi

We solved(?) the hypheantion problem, but got instead a circularity problem. Meeting 12: 30 Norw. time Tuesday to fix the circularity problem after the hyphenation fix.

TODO:

• fix hyphenation of derivations (Thomas, Tomi, Sjur, Trond)
  • done, but not tested
• fix circularity issue (Sjur, Tomi, Trond)

Lule Sámi

Name conversion improvements last week.

TODO:

• lexicalise words from the Olavi missing list, but check against the pdf original where in doubt (Per-Eric)
  • Ilona is working on the last letter in the alphabet, we're almost finished: )
• sme->smj lexicon conversion to build bilingual lexicon resources, and increase smj coverage (Trond, Thomas, Svenne)
  • some additions coming in there as well, but not until we have checked every new word manually
• sme->smj name conversion (Trond, Thomas)
  • done
• add proper nouns (Thomas, Ilona)
  • nothing more to be done
• look at missing baseforms (Thomas)
  • only two missing baseforms at the moment: )

This sub-project needs to get up and running soon. Mainly Sjur's task.

Decisions made in Tromsø can be found in this meeting memo.

TODO:

1. set up Tomcat and risten.no on the G5 again (Sjur, Børre)
   1. install risten.no
2. fix bugs in lexc2xml; add comments to the log element (Saara)
3. finish first version of the editing (Sjur)
4. test editing of the xml files. If ok, then: ( Sjur, Thomas, Trond)
5. make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (Sjur, Saara)
6. convert propernoun-($lang)-lex.txt to a derived file from common xml files ( Sjur, Tomi, Saara)
7. implement data synchronisation between risten.no and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way)
8. start to use the xml file as source file
9. clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (Thomas, Maaren, linguists)
10. merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (linguists)
11. publish the name lexicon on risten.no (Sjur)
12. add missing parallel names for placenames (linguists)
13. add informative links between first names like Niillas and Nils ( linguists)

Hunspell

TODO:

1. Follow-up support for the sami languages in OpenOffice.org (Børre, Sjur)
   1. there is still hope to get them included in OOo 2.4
2. Hunspell lexicon conversion (Tomi, Børre)
   1. nothing major happened last week, working on getting the basic conversion correctly (ie complete and correct inflection), then adding derivation
3. add hunspell testing to the make file (Sjur)
   1. done

Testing

Spelling Error Markup

TODO:

• Set up ways of adding meta-information (source info, used in testing or not, added to lexicon or not) (Saara)
• move Steinar's error markup in the xml files to (a copy of) the original ( Børre, Kimme)
• add nested error markup to xml conversion (Saara)
• test new and nested error markup (Sjur)

Automated testing

Paradigm testing implemented, but the reporting needs improvements. Testing manually:

cd gt/sme/testing
gen-paradigms.sh

Output in paradigm-sme.txt.

Test words should go in the file parawlist.txt. The format is like this:

WORD<TAB>POS
áigi	n
ráhkistit	v

We need to separate proper nouns from regular nouns properly, by checking the files prop-codes.txt and n-codes.txt, as well as making sure the Makefile has support for proper noun paradigm generation.

TODO:

• add hyphenation testing (Sjur)
• add hunspell testing (Saara, Sjur)
  • implemented
• add paradigm testing (Børre, Sjur)
  • done, but needs improvements

MS Office

We need to check that the proofing tools are working in all Office applications, not only Word. This is especially important on Windows, which includes quite a few applications in the bundle.

TODO:

• test the proofing tools with all MS Office applications (Børre, Thomas)

Lexicon conversion to the PLX format

Open issues based on test results:

smj

482 - still problematic, 552 - still open, Svierigadárogielan - still rejected

sme

397 - not as good as it should be, 408 - fixed, 419, 425 - roman number, 431, 452 - miel is a prefix, 461 - ovda accepted, almost 50 % (17) gets correct suggestion, 489, 508 - fixed, 522, 524.

TODO:

• look at test cases still not behaving properly (Thomas, Tomi)

InDesign tools

TODO:

• add hyphenation testing (Sjur)

Hyphenators

We should look into the possibility of generating pattern-based hyphenation for OOo. It shouldn't be too hard, or require too much work, but needs investigation.

TODO:

• get command line hyphenator (Sjur)

Release version

We need to print the CD cover pretty soon, and for that we need to finish the CD cover, especially the back side text. The cover proof print should be ready in 14 days, ie Nov. 26.

TODO:

• try to burn a CD at SD (Risten, Leif-Åge)
• finish text to go on the CD cover (Risten)
• set up CD-printing printer (Risten)
• fix Windows CD installation bug (Sjur, Børre)
• get price and schedule for printed CD cover (Risten)

Actual release

December 11-13, one of these days.

Corpus contracts

Delayed till after final release.

TODO:

• publish corpus contracts and project infra as open-source on NoDaLi-sta ( Sjur)

Faroese

Speller for fao using our infrastructure and the knowledge we have.

TODO:

• set up a telephone meeting with them and Sjur ( Trond)

Bug fixing

When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version.

69 open Divvun/Disamb bugs (35 of these 56 are speller-related bugs, 34 are other bugs), and 23 risten.no bugs

Dictionaries

We have now 7 dictionaries in xml format that could be published in a risten.no-like framework. Trond would like to have them online, both for the feedback and usefulness to the public.

It requires that eXist/XQuery + homepage, and conversion to a proper presentation format is in place.

TODO:

• eXist on G5 - done
• risten.no on G5 - homepage done, but needs port tweaking/changing
• risten.no XQuery framework - NOT done
• XInclude conversion script - NOT done
• homepage on the documetation pages - draft done

Parallel corpora

There will be a parallel corpus seminar this weekend. It would have been nice to have parallel text for that:

• fkvnob (from Kvensk Institutt)
• smenob, smanob, smjnob (from Sámi skuvlahistorjá)

TODO:

• discuss more parallel texts (Børre, Saara, Trond)

SD yearly personell seminar

6.-7. December. Sjur will discuss it with Julia, but our view is that we don't have time to go this year. The release are just a few days later.

Software updates

• SubEthaEdit - Ilona still missing: )
• Leopard, 10.5
  • installed:
    • Børre
    • Sjur
    • Risten
  • not yet:
    • Thomas
    • Ilona
    • Tomi
    • Trond
    • Per-Eric
• Skype 2.6.x

The next meeting is 19.11.2007, 09: 30 Norwegian time.

Trond will be away in the next meeting.

The meeting was closed at 11: 13.

Boerre

• move Steinar's error markup in the xml files to (a copy of) the original
• fix Unicode bug in Hunspell conversion java code
• fix bug 550
• move Bugzilla to the G5.
• fix Windows CD installation bug
• discuss more parallel texts
• fix bugs!

Ilona

• lexicalise smj missing words
• other smj tasks, ask Thomas

Maaren

• lexicalise actio compounds

Per-Eric

• lexicalise words from the Olavi missing list
• fix bugs!

Risten

• finish the design/text for the CD and the cover
• set up CD-printing printer
• try to burn a CD at SD
• get price and schedule for printed CD cover

Saara

• add new XSL/XML headers for proofing test docs
• Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not)
• add nested error markup to xml conversion
• discuss more parallel texts

Sjur

• work on the XML name editor/risten.no integration
• set up risten.no on the G5 again
• test new and nested error markup
• get command line hyphenator for automated testing of the hyph-lexicons
• add hyphenation testing
• improve paradigm testing
• fix bug 550
• follow-up support for the sami languages in OpenOffice.org
• fix Windows CD installation bug
• fix circularity issue in nonrec transducers
• fix bugs!

Thomas

• sme->smj lexicon conversion to build bilingual lexicon resources
• check for bad hyphenation
• look at test cases still not behaving properly
• paradigm testing
• fix bugs!

Tomi

• Hunspell lexicon conversion
• fix circularity issue in nonrec transducers
• fix bugs!

Trond

• sme->smj lexicon conversion to build bilingual lexicon resources
• fix hyphenation of derivations, inflections
• telephone meeting with Sjur and the faroese group re faroese speller
• fix circularity issue in nonrec transducers
• discuss more parallel texts
• fix bugs!.