kimi zamanlar "geyik çevreleri"nde "isn't"ın kelime oyunları içinde kullanıldığına şahit oluyordum, misal "of coursen't" (ofkozınt gibi okuyunuz, belki tutar), ama daha da "orijinal" bir kullanımına bugün denk geldim.
caiz - caizn't!
denemesi bedava!
geçen seferki listeden hareketle:
kapattık kardeşim!
References
Conclusion
This project, while it only consisted of ek$iAPI, had started as an exercise in C#, not thinking that a senior design project could be based on it. The source of inspiration for this graph visualization application was the Skitter(*) project which also featured a circular graph, but laid out in a completely different fashion.
After making the decision of doing this project, a tremendous amount of effort was expended to complete it. However, even more could have been expended for a total fulfillment. Quoting from the Preliminary Report:
“Scope: A detailed inspection of Ekşi Sözlük data in the form of a digraph as a way of representation, with some simple algorithms employed for coming up with the digraph. Extensions, such as marking the titles one specific suser has written, finding cycles of association or creating timelines (or a histogram) of activity for a specific title can also be implemented.”
“The latter and final step is to design and implement the graphing tool which will work on the extracted data. This tool will make use of some simple algorithms or checks. Some are:
- Checking the number of entries under a destination title before assigning a connection between two nodes depicting titles. This will be necessary, as links sometimes are used for other purposes by susers, such as emphasizing a part of the entry. Also, some links point to non-existent titles which should be eliminated.
- Possibly, a node distribution algorithm, so that no node of the graph overlaps with another to allow clarity of presentation.”
ek$iVista
The Problem: Designing a program that generates a digraph from the edge data generated by ek$iEdgeDump.
Design: This part of the project was the most troublesome part, involving a cascade of decisions. As the project supervisor, Prof. Attila Gürsoy advised making use of graph layout libraries, such as JUNG(*) (Java Universal Network/Graph Framework), yWorks(**) or GraphViz(***). During the research phase, however, it was observed that neither of these options was worth the effort; JUNG could not be used within C#, yWorks was a commercial package with costs beyond my budget for the foreseeable future, GraphViz seemed too hard to implement. Later on, I found a Windows DLL port of GraphViz(****) and modified ek$iVista to generate a text file in DOT format (file format accepted by GraphViz). However, the lexer inside GraphViz could not parse the text file generated by ek$iVista with no apparent reason, so the use of the GraphViz was out of question. It was becoming obvious that the layout algorithm and generation of the diagram had to be handmade.
From a very large array of layout algorithms, like the Kamada Kawai algorithm, a random vertex-placement algorithm and many tree layout algorithms, the circle layout was chosen, because in this layout, the vertices were pushed near the borders of the diagram and the center part was left vacant for the edges to be placed. Also, the coordinates of vertices and edges could be calculated by simple trigonometry. First of all, a minimum distance between two vertices is defined; let us name it md. If there are v vertices laid out evenly on the perimeter of a circle, the perimeter is expected to be roughly (md x v) units. The radius of the circle, hence, is (md x v)/2π. The position of the nth vertex on the diagram, given the center of the diagram as the Cartesian pair (cx, cy) is the Cartesian pair (cx + cos(360n/v)((md x v)/2π), cy + sin(360n/v)((md x v)/2π)). As we know the coordinates of the vertices and have a list of edges, drawing the directed edges should be trivial.
Everything is expected to fit in without any problems, but expectations are not always met. For enabling interactive vertices that respond to clicks, a control named VistaVertex was created which contained four buttons envisaged to fire some events and methods. However, as the number of vertices rose, the application became more than cumbersome. Also, an unadvertised “feature” of Windows surfaced; one cannot create more than 10,000 controls per application, because Windows cannot generate “handles” for them. Because of this limitation, the use of controls was impossible; the diagram had to be painted on the form and it could not be interactive for the time being. During this phase, I had mounting difficulties when the form had to be refreshed, because whole diagram had to be painted from scratch and I could not figure out a method to avoid this. Finally, I decided to generate a viewable image by using the graphics libraries provided in C#, and discovered another hidden limitation; drawing images larger than 32,678 x 32,768 was impossible. Such a limitation was also imposed upon the size of forms; although the property that keeps the height and width of the form is of type Int32 (232 ≈ 4 x 109), the maximum value it accepted was 215. With the current number of vertices, however, this poses no big problem.
The program first reads the source titles into a Hashtable and gets the total number of vertices. After this, a SortedList (a Hashtable sorted according to the keys of the items) object is populated by Point objects that store the calculated coordinates of the vertices with the title names assigned as their keys. Then, EksiEdgeData table is read from beginning to end; if the source title corresponds to a key in the hashtable, the directed edge is drawn. After all the edges are drawn, the vertices are drawn onto the image, and finally, the image is saved at a fixed location, the root of the C:\ drive.
The graph drawn by ek$iVista, although substantially rich in data, is not very adept at displaying the connections between Ekşi Sözlük titles as much of the meaning is lost in the clutter. As it was stated before, the “final” graph produced contained the details of only a small portion of Ekşi Sözlük data and finally, the graph, although envisaged to be interactive at first, was far from interactivity. To rectify these shortcomings, a new version for ek$iVista was
The Problem: Providing means of visualizing and analyzing Ekşi Sözlük data gathered by ek$iDump – especially the links between titles and the users contributing to titles. Also, correcting the flaws of the first version; trying to draw the whole graph which makes it unintelligible, having to rely on a separate table (EksiEdgeData) generated beforehand to come up with the graph while the data for it could be generated on-the-fly, and providing no outlets for interactivity.
Design and Implementation: The first design decisions were about what to include in this application and what to leave out. To see what has been done clearly, let us use a weekly update mail as our checklist:
“I spotted a graph visualization package named Netron (http://netron.sourceforge.net/) and will be using this package for the title connections graph.”
- “The queries that I am going to use in ek$iVista are:
- The one that will be used to draw the connectivity graph (with the option of displaying titles 1, 2, 3, 4 and 5 clicks ahead)
- Simple queries that will list the users who contributed to the title and the entries under the title (with the option of opening it from the database with or directly from Ekşi Sözlük)
- Queries that will help to draw timelines for activity, for titles and users
- A query for finding the "intersection set" of the titles written to by two distinct users”
ek$iEdgeDump
The Problem: Extracting links (connections) from the existing “heap” of entries.
Design: For ek$iVista to be able to function, to be able to produce a digraph, it needs a list of directed edges, and ek$iEdgeDump was produced for this purpose. For the sake of simplicity, like ek$iDump, it is also designed as a console application. During the development, two versions of ek$iEdgeDump were produced. The first version gets the full list of titles in the database (a table named Titles exists in the database) and scans them one by one. In this scan, the entries under the title being scanned are inspected and any link that points to a title (those that point to single entries are omitted) is parsed out of the entry text. Then, another database query checks whether the title pointed by the link exists in the title list. If it exists, the pair consisting of the IDs of the source title and the destination title (the records in the Titles table have a title ID and title name) are written to a table named EdgeData, only to be used by ek$iVista in drawing the digraph. This approach proved to be too slow, because for every link found in an entry, a verification query has to be made. The scan rate of this version of ek$iEdgeDump was less than 1,000 titles/day. Given the fact that the database contained more than 700,000 titles, the job would be completed in nearly two years. Clearly, another approach had to be adopted .
In the second version of ek$iEdgeDump, the focus is back on the entries instead of the titles. As one can recall from the description of the Entry class in ek$iAPI, one of the details of acquired from Ekşi Sözlük when an entry is extracted is the title the entry is placed under. Thus, we can produce a different table that looks like the EdgeData table described above that keeps information of the source and the destination vertices of the directed edge. The table, in the new approach, is produced by scanning the entries in the database (they reside in a table named Entries), parsing out the links that point to titles and writing the pair consisting from the name of the source title and the destination title to a table named EksiEdgeData without checking whether the destination title exists in the Titles table. This verification effort was the factor that slowed the first version down, and it can be handled without querying the database by ek$iVista (the details of how this is done are given in the section discussing ek$iVista). As the title data in the Entries table is stored in string format (not as integers; foreign keys related to title ID column in Titles table), the size of the EksiEdgeData table is significantly larger than that of EdgeData. ek$iVista uses the data from EksiEdgeData table, generated by the last version of ek$iEdgeDump.
ek$iDump
The Problem: Coming up with a portable application to acquire Ekşi Sözlük entries and store them in a database.
Design: Although the final product of the project will be a graph depicting connections between Ekşi Sözlük titles, the connections arise from the content of the titles, which are, obviously, the entries. That is one of the reasons why ek$iDump is a tool for getting the entries rather than the titles. Another and maybe the prime reason for focusing on entries is that entries have unique integer IDs that allow them to be acquired one by one in a for-loop or a while-loop.
ek$iDump is, due to this nature of Ekşi Sözlük entries, at the level of complexity of a “Hello World” program. The program, designed as a console application, gets the starting ID and the terminal ID as its input, which are integer values. In a while-loop, beginning from the starting ID, if the entry with the given ID exists, it gets it from Ekşi Sözlük by calling Entry.GetFromEksi(ID) and writes the details of the Entry acquired to the database. The database of choice is a Microsoft Access file, because one does not have to set up a server for using it; even if you do not have Microsoft Access installed, one can obtain and install a package named Office 2003 Redistributable Primary Interop Assemblies and get on with using the database. Also, the data accumulated in the database is easily exportable to Microsoft SQL Server, which the other two sections of the project, ek$iEdgeDump and ek$iVista use. One always has to make the quantum leap from Microsoft Access to another pro-level RDBMS at some level, as Microsoft Access imposes a size limit of 2 GB on a database file. Note that although it is by no means final, the size of the database file generated in the course of the project exceeds 5 GB. An image showing ek$iDump in action is given in the figure below:
ek$iAPI
The Problem: Retrieving information from Ekşi Sözlük and organizing it programmatically.
Design: In the very beginning, the plans were basically getting started with ek$iDump and incorporating data acquisition functions into some method inside the main class. That would have been very easy to start with, however, if I wanted to use the portions of code that access Ekşi Sözlük in another application, all had to be re-written. To avoid such circumstances, a different style of programming had to be adopted. After some contemplation, I decided to program this part of the project as a class library. Class libraries are collections of classes and methods inside classes, and when compiled in Visual Studio.NET, are built into Windows DLLs. Thus, I would be able to reuse the code in any application.
Although it carries no formal significance, ek$iAPI did spring from the drawing board:
Project Divisions
Ekşi Sözlük graph visualization project consists of four distinct applications:
What is Ekşi Sözlük ?
Ekşi Sözlük, founded on February 15th, 1999, is a site in Turkish which can be classified as a “collaborative hypertext dictionary”; the content is formed by its more than 10,000(*) active users. The users prefer to call themselves in many different ways; susers (sözlük users), “sözlükçü”s or “yazar”s (writer, author). Users, according to their membership dates, are divided into generations:
1st generation: Member as of February 1999
2nd generation: Member as of May 2000
3rd generation: Member as of May 2001
4th generation: Member as of May 2002
5th generation: Member as of April 2003 (after a book donation campaign)
6th generation: Member as of May 2004
7th generation: Member as of November 2005 (after a book donation campaign)
Information in Ekşi Sözlük is organized under titles (tr. başlık) which can be potentially about anything one can think of; some are meant to be informative, some are hilariously funny, while many of them are generally silly or weird and for reasons unknown, are limited to 50 characters. Although the site is in Turkish, the six letters that are not in the English alphabet (ç, ğ, ı, ö, ş, ü) are not used in the titles. However there is no such limitation for the entries. What makes Ekşi Sözlük a hypertext dictionary are the cross-references, named “bakınız” and abbreviated as bkz (translatable as “see”). There are several types of cross-references, and they are:
Introduction
The first steps leading to this project were taken in Fall 2004 semester, in MGIS 302 (Software Programming Techniques) course given by Ömer Yedekçioğlu. As the term project, I had presented a program written in Visual Basic 6 to back up Ekşi Sözlük (http://sozluk.sourtimes.org/) entries in an HTML file, which was aimed at being a replacement for SourLemonade (http://213.232.33.34/sourlemonade/default.aspx), the “official” service for Ekşi Sözlük users to backup one’s own entries. The program, dubbed “ek$iBackup”, however, had no such limitation; one could backup the entries of any user (s)he wished to backup. Also, after all the entries by a particular user are scanned by ek$iBackup, it allowed viewing of entries and the titles that the entries pertain to either from the backup or directly from Ekşi Sözlük.
hani proce raporu diyorduk ya, işte ta kendisi. ancaaaaak, önce neyin üzerine rapor yazıyoruz bilelim, di'mi?
i kept on telling about some project report, and there it is. but first of all, we have to know what this report is written about, innit?
egiboy planlama teşkilatı günlük hayat gözden geçirme raporu #1! başlığa bak hizaya gel. neysemkine, plan program ama illa ki geyik olsun deyu bir liste hazırlayalım. bakalım ne yapmışız, akabinde neleri yapıcaz. ahanda:
şimdilik buğadder.
superkalifracilistik bir arama sürecinden sonra (anadolu hayat'ın 2 aylık ve olumlu sonuçlanmasını son mülakattan beri beklemediğim (ve beklentimde haklı çıktığım (metaparentezlere başladım, süper olay)) harikadan da öte eleman seçme süreci dahil) gavurun deyişiyle "decent" bir yerdeyim. diyeceksiniz ki neresi? ahanda burası:
kendileri bir eğitim kurumu olmaktan daha da fazlası, aynı zamanda outsourcing falan fiştan da yapmakta. onlar beni c# ve java biliyorum diye seçtiler, bir de pl/sql ve javascript kassam tam olacak gibin.
törenden başlayalım. geçen seneki gibi mezun başına 32 kentilyon (quintillion) saniye konuşma süresi tanınmamış olması çok önemli bir gelişme. hatta konuşmaları tamamen elimine edip video kayıtları ile törene ayrı bir renk yaratmış olmak da takdire şayan. tören konusunda "ben harvard'ı gördüm, yale gördüm, koç lisesi robert falan da gördüm, hiç bu kadar güzel bir mezuniyet töreni görmedim" diyen, yanılmıyorsam dinçkök ailesi mensubu hanımefendi konusunda ise "abartmış biraz" ya da "yo kartık daha neler"den başka hiçbir şey diyemiyorum. "land of hope and glory"'siz mezuniyet töreni mi olur be :P
neyse, ahanda törenden bir resim:
organizasyon konusunda okulu kınıyorum ve kendilerine laflar hazırladım(hınzır smiley was here). öncelikle, organizasyonsuzluktan bıktım. bık-tım. son anda bir "okulda yer ayarlayamıyoruz, herkes muallimköy'deki öğretmenevine rezervasyon yaptırsın" deniliyor, bir de aranıp öğreniliyor ki öğretmenevinde yer yok. kalmak isteyenlerin okulda kalacağı, daha doğrusu kalmak zorunda kalacağı" tebarüz etti" tabii ki. bir sürü insan da sırf okulda kalamayacağı için gelmedi, gelenlerin önemli bir bölümü de tören sonu gerisin geri nereden geldilerse oraya döndüler ve ertesi günkü "mezunlar günü"ne teşrif etmediler. teşrif etmeyenlerin ne kadar isabetli bir karar verdiklerini izleyen satırları okuyarak anlayacaksınız.
bu noktada okul yönetimince anlaşılması gereken şey şu; mezunlar okulda kalmaya geliyorlar, ama bu "kalma", yönetimin tahmin ettiği şekilde geceleme, barınma, okula hostel muamelesi yapma manasını taşımamakta, daha çok yeni mezunlarla kaynaşma, en tatlısından, paylaşılan anılar ve havada uçan esprilerle süslü bir "inisiyasyon seremonisi" şeklinde geçmekte. bu yıl, yeni mezun elemanlar hemen "tumba yatak" emri ile erkenden mayıştıkları için bu sene bu da pek mümkün olamadı.
sonrasında o 9 ay öncesinden planlanmış muazzam mezunlar günü geldi çattı. çattı da ne oldu? HİÇBİR ŞEY. abuk voleybol-futbol-basketbol müsabakaları ile 4 saatlik süper organizasyon pek de iz bırakmadan tamamına erdi. bir de amacına ne kadar ulaştığı benim için hala muamma olan bir panelimsi kümeleşme vardı. orada da "dernek kuruyoruz, ha kurduk ha kuracağız" minvalinde açıklamalarda bulunduk, bir şekilde kaybolacağından emin olduğum bir kağıda diğer katılan mezunlarla beraber iletişim bilgilerimi yazdım. inşallah telefon numaramı bağcılar'da bir üst geçitin ayağında "bel fıtığı" yazısı eşliğinde görmek zorunda kalmam. biraz "strateji oyunları" kısmında, tabu (bildiğiniz tabu, hani şu tabu var ya, o tabu) adlı strateji(?) oyununda pop-kro kardeşlerime (aemin ve yılmaz yandaşları) yenilirken eğlendim işte, o da o kadar. günün en önemli artısına da değinmezsem ayıp olur; lise 2'lerden ali cem ile akşam saat 10'da kola meşrubat falan içesimiz tuttu, gittik rasim bey'e dedik "böyle böyle". aynen öyle. o da "değişik bi'şi' yapalım" diyerek aldı bizi eskihisar'da bir çay bahçesine götürdü. okul hakkında konuşarak geçen gayet güzel bir iki saat yaşadım, teşekkürü borç biliyorum kendilerine. yine de alamadık kolayı, iyi mi :)
soonacııma ertesi günün sabahında okulun taze binalarının fotolarını çektim, o da şöylemesine bir şey:
tekne turu falan derken deli yoruldum, bir sürü boş beleş foto çektim (gidin bi' flickr'ıma bakın, hak vereceksiniz), güzel insanlarla tanıştım, daha ne ola? ne bileyim, mesela gelecek yıl daha güzel bir mezunlar günü ola!
