Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1117 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.1838. Min Act: -0.1700

Data Index: 1290081 (Open Web Text)

Max Activating Token Index: 315

Text #1

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.1557. Min Act: -0.1700

Data Index: 7495847 (Open Web Text)

Max Activating Token Index: 187

Text #2

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0417. Min Act: -0.1700

Data Index: 1956857 (Open Web Text)

Max Activating Token Index: 471

Text #3

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0338. Min Act: -0.1700

Data Index: 6531761 (Open Web Text)

Max Activating Token Index: 36

Text #4

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0287. Min Act: -0.1700

Data Index: 3477306 (Open Web Text)

Max Activating Token Index: 441

Text #5

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0207. Min Act: -0.1700

Data Index: 5107701 (Open Web Text)

Max Activating Token Index: 47

Text #6

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0267. Min Act: -0.1700

Data Index: 8360879 (Open Web Text)

Max Activating Token Index: 626

Text #7

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9881. Min Act: -0.1700

Data Index: 487618 (Open Web Text)

Max Activating Token Index: 98

Text #8

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9911. Min Act: -0.1700

Data Index: 6122238 (Open Web Text)

Max Activating Token Index: 279

Text #9

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9908. Min Act: -0.1700

Data Index: 1562893 (Open Web Text)

Max Activating Token Index: 792

Text #10

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9896. Min Act: -0.1700

Data Index: 6358121 (Open Web Text)

Max Activating Token Index: 788

Text #11

Max Range: 4.1838. Min Range: -4.1838

Max Act: 4.0296. Min Act: -0.1700

Data Index: 7778399 (Open Web Text)

Max Activating Token Index: 869

Text #12

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9671. Min Act: -0.1700

Data Index: 8256575 (Open Web Text)

Max Activating Token Index: 607

Text #13

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9708. Min Act: -0.1700

Data Index: 5921421 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9907. Min Act: -0.1700

Data Index: 8042285 (Open Web Text)

Max Activating Token Index: 382

Text #15

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9309. Min Act: -0.1700

Data Index: 2925164 (Open Web Text)

Max Activating Token Index: 377

Text #16

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9271. Min Act: -0.1700

Data Index: 5322873 (Open Web Text)

Max Activating Token Index: 771

Text #17

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9266. Min Act: -0.1700

Data Index: 3767343 (Open Web Text)

Max Activating Token Index: 873

Text #18

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9249. Min Act: -0.1700

Data Index: 6043139 (Open Web Text)

Max Activating Token Index: 880

Text #19

Max Range: 4.1838. Min Range: -4.1838

Max Act: 3.9220. Min Act: -0.1700

Data Index: 4408029 (Open Web Text)

Max Activating Token Index: 960