[Lvlug] askSam/TWiki Clone (Project?)

[Randy Kramer]

I replied rather quickly to this earlier and didn't really look at the 
Beagle/Dashboard/etc. stuff.  So...

On Monday 29 August 2005 05:13 pm, Pat Regan wrote:
> I have been toying with some thoughts about building a wiki based around
> Darcs:  http://abridgegame.org/darcs/

Why Darcs?

> I have no idea what experience you have with revision control systems...
>  Darcs allows for very easy branching and merging.  Most wikis that are
> worth anything have some sort of revision control.

TWiki certainly does, and I think I have a fair understanding of rcs's and the 
concepts at a user (as opposed to a frequent user or developer) level.  On 
many of the projects that I was involved with (more at a project engineering 
level), I've successfully pushed to have tools like that used.

> Are you sure you are looking to create just a local wiki though?  

No, absolutely not!  

That's some of the stuff I haven't discussed yet.  It's a little hard for me 
to describe what I'm looking for in a few words, and because I didn't want to 
disclose it all--after I spend more time looking at emacs wiki, if they 
already do it all, there'll be no point in being shy about disclosing more.

But (to disclose some more ;-) another part of what I want is a combination of 
local and global (i.e., Internet hosted) wikis.  Local for fast access and 
update and for private stuff, global to share with others (and encourage them 
to share).  Because I'm so familiar with TWiki, I planned on starting by 
using TWiki as a "back end", at least for the Internet wiki.  

There are some things I'd like to improve in TWiki, and if those haven't been 
done by the time I get some other things done, I'd try to address those.  
(One is better support for multilevel hierarchical webs (i.e., 
folders/directories), but they're actually working on that now.  Another is 
word style diffs.  I've had some other thoughts, would have to look for a 
list (I'm sure I have one ;-)  (TWiki is written in Perl though, and at one 
point I tried learning it and gave up.  I think I'd do better if I tried 
again, but part of me thinks I'd prefer to rewrite it in another language 
(partially as part of the "plan to throw one away" school of design).

(Part of my problem at the time was regular expressions, I've since got a 
better handle on those and have encountered (and successfully dealt with 
them) in other languages.)

> Your 
> ask sam claims to be a "free form database."  What exactly is missing
> from your run of the mill wiki that askSam can do?

Hmm, I'll have to try to remember:

   * askSam has much better search methods--I've "improved" TWiki's somewhat 
(on WikiLearn) by including a Google search thingie, but askSam had things 
like what I will call proximity searches, i.e., one word within x words (or 
same sentence, paragraph, ...) of another

   * askSam made it much easier / quicker to enter data.  With a(n askSam) 
database open in an askSam window, just go to that window, click new (or 
similar, I actually forget) and start typing.  You can start typing 
"instantly".  You can save (and then refind) that record again without giving 
it any sort of title (the "title") is implicitly the first words you type, 
which you can change at any time to make a better (more relevant/more easily 
findable) title.

   * even though askSam records are basically free format, you can structure 
the data by creating fields (at any time, and in some or all records).  TWiki 
(to the best of my knowledge after trying to make a pretty thorough search 
about 5 years ago, and trying to pay attention since then) is the only wiki 
that has any similar capability--I'm trying (without going back to look) to 
recall what TWiki calls it, but iirc, it is their form fields.  It seems 
somewhat more cumbersome, partially because you have to create a(n HTML like) 
form in advance (or more than one, but can only apply one per record) and 
then are limited to the fields defined in that form.  In askSam you can 
define a field at any time by simply typing:

:<field name>[ <filed content> ] 

(I'm sure there are some details I'm forgetting about this.)

The first thing I'm trying to address with the "project" is allowing me to 
create new records "instantly" (by allowing me to create and save them 
locally and then queing them for update on the "server").  (Of course, I have 
to address things like duplicate records, editing conflicts (because I plan 
to allow local editing of existing records as well as creation of new 
records), and so forth.  I have "plans" (ideas ;-) on how to deal with these.

I might mention that askSam, at least in some versions, included a builtin 
word processor with the intent that any documents you created with it were 
immediately incorporated in the askSam database.  I didn't find that very 
useful as it was not directly compatible with Word, nor did it use some of 
the tools that I was very used to from Word (styles for formatting, 
collapsible outlining, automatic TOC and indexes, ...).   So far, a similar 
capability is not on my "todo" list although I do expect to have things like 
email and snail mail correspondence indexed.  (Beagle looks like a good 
solution for that.)

> Have you seen Dashboard and Beagle?

Nope, or not until now, and they look neat--something else I'll spend a little 
time looking at.  

>> http://www.nat.org/dashboard/
> http://beaglewiki.org/Main_Page
>
> Beagle is a sort of realtime search facility, it is kept up to date
> using the new inotify kernel facility.  Dashboard uses inotify as well.
>  It watches files that you are working on and constantly searches for
> things related to what you are working on.  There were video
> demonstrations of Dashboard a while back, but I didn't see them on the
> web site.
>
> I am not sure exactly what itch you are attempting to scratch, but I
> have a feeling those two projects may do a lot of what you want from an
> entirely different angle.

Yeah, and a little bit like the way I used to use ZyIndex (to take all 
(computer generated/based) correspondence and documentation on a project and 
index it for quick searches).

I will get on my soapbox for a minute.  I like wikis / askSam for a lot of 
reasons, but among them is the idea of "cooking" the contents.  What I mean 
by that is, an email archive or a web forum is much less useful to me, 
because, for example, if there is a thread (or more than one thread, maybe on 
more than one mail list), a search will often bring up all the posts in that 
thread, then I have to read (skim) or re-search (if I can think of a limiting 
search term) to find the nugget(s) of information for which I'm looking.

With a wiki, I can "cook" the information in various ways--paraphrase (or 
quote (with attribution in case I later want to "publish" the information)) 
or simply save the link with a description of why I thought the information 
is (or might be) useful.

(I guess part of the paradigm that I didn't explicitly state is that I (like 
most other people) encounter information that I think will be useful at some 
point but maybe not at this instant.  So, I want to save that information (or 
a link to it, or whatever) in a form that will make it easily findable and 
usable when (if) I do need it.

By the way, for that reason, I  like the "Stuff I've Seen" name for the one 
Microsoft product--if there's something I've seen that might help me in some 
way, when I need it I'd like to find it again.

(I could get into a long philosophical discussion here (maybe I already did), 
but a Google search that returns 100,000 hits that I can't narrow down often 
leaves me with a lot of work to do before I find what I'm looking for.)

I'll mention one other feature I eventually planned.  I want to have an 
automatic email copy/paste/edit feature.  It would work like this--if I see 
an email I want to save (possibly one or more of the nuggets from a long 
thread), I'd select and copy it, then paste it into the database.  On 
pasting, the database program would automatically clean up some of the 
formatting, for example, if (as usually happens for me), extra newlines are 
inserted in and between the To, From, Subject, and Date headers, the program 
would remove those.  Similarly:
   * if there is quoted text in what I've copied, it would either apply the 
TWiki markup for <pre></pre> so that proper line wrapping (ie., none) 
occurred (or, if it seemed usable, reformat the quoted portions (typically 
prefixed with ">" or similar so each paragraph was prefixed with that mark, 
but the lines within the paragraph wrapped (reflowed??) properly within the 
current width of the window.
   * If my mail client took some real dates and replaced them with things like 
"today", "yesterday", "Saturday" (for the last Saturday) and replaced them 
with the real date.  (In the future, what use is it if all the emails in the 
database are dated "today" ;-)

> Anywho, I have lots of free time so I would likely be willing to help
> you out if we can flesh out what the actual goals of the project will be.

Hold that thought (please)!  I do want to dig deeper into the emacs wiki, and 
maybe I'll adopt that and move on, but I do think there is a 
"market" (demand, although perhaps a "latent" demand) for something with the 
features I envision but with a much better user interface than emacs.

In fact, I'd put it to you this way, if you agree, and are willing to put in 
some time, let's get started anyway (we can always stop).

There will be one other thing (and maybe I've dsiclosed too much already), but 
if we start down this path, I would like at least some sort of informal 
non-disclosure / non-compete agreement (at least until we actually release a 
project, or mutually agree that the project is dead and I have no further 
plans to resurrect it.  If that worries you, let's talk about that.

I do have various (partial) descriptions of the project.  Most recently, I 
started working on a description of the 2nd alternative that I mentioned in 
that earlier post (I forget how I described it, but the one where we 
basically build the project into something like khtml and call a nedit client 
as the external editor).

I think I'm going to spend a few days (or longer :-( ) looking at the emacs 
wiki and maybe trying to get it going for me locally.

I know I have another post from you, I'll look if over--not sure it needs a 
response more than what's already covered here.

In any case, thanks for your interest so far!

regards,
Randy Kramer