A PhD student's perspective on research in NLP in the era of LLMs

[rhdunn]

I've been interested in NLP for tagging stories based on topics and themes (detectives, werewolves, murder mystery, etc.), so need accurate disambiguation of parts of speech and ways of detecting uses of metaphore, similies, etc. to describe those. I also want to be able to assess how much of the text is about a given topic, so that if I'm interested in reading a detective story from e.g. the Project Gutenberg collection, I don't want it to pick up a story where a detective is only mentioned in one paragraph.

I've looked at several existing NLP frameworks (Open NLP, Stanford NLP) and none of them are accurate enough -- they fail on things like adjectives and old english second person pronouns. This makes them practically unusable for proper sense diambiguation, lemma and part of speech based rules, etc.

The Open NLP tokenizer is also terrible at tokenizing title abbreviations ("Dr", etc.) and things like the use of "--" to delimit text, which is frequently found various Project Gutenberg texts. You can train the Open NLP tokenizer, but it works on what it has seen, so you need to give it every variation of "(Mr|Mrs|Miss|Ms|Rev|Dr|...). [A-Z]" for it to tokenize those titles; the same for other tokens.

[nl]

Have you tried Spacy?

I find it substantially better than other tools as PoS tagger.

Also worth noting the that your assertion that you need these features to classify genres isn't obviously true to me at all.

[viksit]

at this point the gpt api will do all of that for you! or you can use the embeddings api and build your own systems.

[dontupvoteme]

Are papers becoming blogs?

[teruakohatu]

It has 10+ pages of references. More like a well researched lit review than a blog post with a few links.

[codethief]

I'd say the culture of scientific publishing precedes blogs by "a few" decades. There have always been papers that were blog-like in nature.

[gumby]

Consider the letters section of Phyicical Review, that turned into a weekly journal Physical Review Letters back in the 50s. It was a blog or forum back when papers were distributed on paper.

Basically because of the slow pace of review and publication the letters column became a way to talk about recent results or problems, and then follow up letters (i.e. comments on the blog posts) became common. So the editors decided to hive it off and speed up its publication schedule.

[sdwr]

Thank **ing god! Hiding behind jargon and "the process" is an indicator of having nothing to say. I see this as a rolling up of the metaphorical sleeves, a sign that stuff is actually happening.

[SecretDreams]

Always have been.

[penguin_booze]

Yeah, wait for them to be twitter threads: 1/n - where (n >> 1).

[vpastore]

Only the one on arxiv. People shouldn’t take anything coming from that site seriously

[dcl]

A PhD Student's Perspective... ~20 authors.

[SecretDreams]

It takes a whole village (of uncredited MASc students) to get a PhD!

[est31]

The particular headline pattern is used a lot and 99% of the time it indeed is at the front of opinion pieces written by individuals. But this time, it's different. In the syntax, in this specific form, the plural and the singular come out the same. E.g. in the sentence "I accompanied my friend to his parent's house", it can be either the house of his single mom/dad or the house of his two parents living together.

[reesul]

Title is off yeah, but the abstract does effectively say that the paper was written amongst a cohort of PhD students

[totorovirus]

well I see topics like NLP in ethics, healthcare, etc, which I think is a sign they don't have much to do here.

[whatyesaid]

Yeah those aren't good especially stuff like healthcare a PhD student can't get the needed data.

The other issue is, if you do focus on LLMs, it's too hyped your research would be too overlapping/competing especially as you've got a dissertation to write. It's a hard problem.

[teleforce]

According to the article, the original research on language model was kick started by Claude Shannon's early contributions in Markov chain model of English words.

If you are in the field of Information and Communication Technology (ICT) there are hardly any area in the field which their fundamentals do not have Shannon's hands in it.

Leonard Kleinrock once remarked that he has to focus on the exotic queuing theory field that later leads to the packet switching and then Internet because most of the fundamentals problems in electrical and computer engineering (older version of ICT) have already been solved by Shannon.

[etamponi]

Isn't the main problem with NLP research now that you'll need a ton of money to run your experiments? How can an "average" PhD researcher hope to validate their hypothesis if they need several thousand dollars per test?

[nl]

No.

There are plenty of research directions that are outlined in this document that don't require huge compute budget.

[jasmer]

Much of the interesting pure research can be done at smaller scale, the larger models are arguably more product engineering than research. At least from a certain perspective.

[Randomizer42]

That is not a problem. There are literally thousands of ways to conduct any experiment at the fraction of the cost.

[al__be__rt]

this paper doesn't appear to have been edited by any reputable journal, so take its authenticity at face value...

[lgessler]

I've got some bad news for you about how papers published at NLP conferences are reviewed.