Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3065 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.4134. Min Act: -0.1700

Data Index: 3562089 (Open Web Text)

Max Activating Token Index: 1022

Text #1

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.2301. Min Act: -0.1700

Data Index: 3485620 (Open Web Text)

Max Activating Token Index: 893

Text #2

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.2033. Min Act: -0.1700

Data Index: 2433363 (Open Web Text)

Max Activating Token Index: 576

Text #3

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.2252. Min Act: -0.1700

Data Index: 745000 (Open Web Text)

Max Activating Token Index: 490

Text #4

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1472. Min Act: -0.1700

Data Index: 3123310 (Open Web Text)

Max Activating Token Index: 582

Text #5

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1810. Min Act: -0.1700

Data Index: 230707 (Open Web Text)

Max Activating Token Index: 653

Text #6

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1439. Min Act: -0.1700

Data Index: 6358538 (Open Web Text)

Max Activating Token Index: 892

Text #7

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1551. Min Act: -0.1700

Data Index: 3563423 (Open Web Text)

Max Activating Token Index: 293

Text #8

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1729. Min Act: -0.1700

Data Index: 5285297 (Open Web Text)

Max Activating Token Index: 379

Text #9

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1597. Min Act: -0.1700

Data Index: 5272355 (Open Web Text)

Max Activating Token Index: 672

Text #10

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1897. Min Act: -0.1700

Data Index: 4268555 (Open Web Text)

Max Activating Token Index: 464

Text #11

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1807. Min Act: -0.1700

Data Index: 4139907 (Open Web Text)

Max Activating Token Index: 836

Text #12

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.2003. Min Act: -0.1700

Data Index: 153016 (Open Web Text)

Max Activating Token Index: 419

Text #13

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1707. Min Act: -0.1700

Data Index: 507190 (Open Web Text)

Max Activating Token Index: 993

Text #14

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1299. Min Act: -0.1700

Data Index: 4252383 (Open Web Text)

Max Activating Token Index: 387

Text #15

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1465. Min Act: -0.1700

Data Index: 1560266 (Open Web Text)

Max Activating Token Index: 405

Text #16

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1467. Min Act: -0.1700

Data Index: 93239 (Open Web Text)

Max Activating Token Index: 202

Text #17

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1379. Min Act: -0.1700

Data Index: 2760661 (Open Web Text)

Max Activating Token Index: 599

Text #18

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1504. Min Act: -0.1700

Data Index: 3749362 (Open Web Text)

Max Activating Token Index: 997

Text #19

Max Range: 4.4134. Min Range: -4.4134

Max Act: 4.1560. Min Act: -0.1700

Data Index: 2020869 (Open Web Text)

Max Activating Token Index: 453