Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2291 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.7572. Min Act: -0.1700

Data Index: 6943837 (Open Web Text)

Max Activating Token Index: 456

Text #1

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.6293. Min Act: -0.1700

Data Index: 1325287 (Open Web Text)

Max Activating Token Index: 935

Text #2

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5703. Min Act: -0.1700

Data Index: 4042454 (Open Web Text)

Max Activating Token Index: 27

Text #3

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5710. Min Act: -0.1700

Data Index: 7143259 (Open Web Text)

Max Activating Token Index: 207

Text #4

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5868. Min Act: -0.1700

Data Index: 5930240 (Open Web Text)

Max Activating Token Index: 305

Text #5

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5572. Min Act: -0.1700

Data Index: 7130258 (Open Web Text)

Max Activating Token Index: 98

Text #6

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5362. Min Act: -0.1700

Data Index: 7837471 (Open Web Text)

Max Activating Token Index: 1000

Text #7

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5200. Min Act: -0.1700

Data Index: 2637795 (Open Web Text)

Max Activating Token Index: 130

Text #8

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5197. Min Act: -0.1700

Data Index: 7248893 (Open Web Text)

Max Activating Token Index: 824

Text #9

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.5164. Min Act: -0.1700

Data Index: 2300767 (Open Web Text)

Max Activating Token Index: 801

Text #10

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4748. Min Act: -0.1700

Data Index: 3605700 (Open Web Text)

Max Activating Token Index: 51

Text #11

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4348. Min Act: -0.1700

Data Index: 5475306 (Open Web Text)

Max Activating Token Index: 966

Text #12

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4584. Min Act: -0.1700

Data Index: 1099384 (Open Web Text)

Max Activating Token Index: 378

Text #13

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4410. Min Act: -0.1700

Data Index: 4602641 (Open Web Text)

Max Activating Token Index: 293

Text #14

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4364. Min Act: -0.1700

Data Index: 6743505 (Open Web Text)

Max Activating Token Index: 820

Text #15

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4253. Min Act: -0.1700

Data Index: 1427737 (Open Web Text)

Max Activating Token Index: 159

Text #16

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4119. Min Act: -0.1700

Data Index: 7147079 (Open Web Text)

Max Activating Token Index: 536

Text #17

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4355. Min Act: -0.1700

Data Index: 5781112 (Open Web Text)

Max Activating Token Index: 427

Text #18

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.3742. Min Act: -0.1700

Data Index: 4782908 (Open Web Text)

Max Activating Token Index: 23

Text #19

Max Range: 4.7572. Min Range: -4.7572

Max Act: 4.4204. Min Act: -0.1700

Data Index: 5826053 (Open Web Text)

Max Activating Token Index: 805