Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2393 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.9541. Min Act: -0.1700

Data Index: 6167870 (Open Web Text)

Max Activating Token Index: 605

Text #1

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.8263. Min Act: -0.1700

Data Index: 4927268 (Open Web Text)

Max Activating Token Index: 634

Text #2

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.8087. Min Act: -0.1700

Data Index: 1474736 (Open Web Text)

Max Activating Token Index: 35

Text #3

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.8393. Min Act: -0.1700

Data Index: 5236729 (Open Web Text)

Max Activating Token Index: 415

Text #4

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7888. Min Act: -0.1700

Data Index: 3450680 (Open Web Text)

Max Activating Token Index: 614

Text #5

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7655. Min Act: -0.1700

Data Index: 1461419 (Open Web Text)

Max Activating Token Index: 294

Text #6

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7782. Min Act: -0.1700

Data Index: 7445847 (Open Web Text)

Max Activating Token Index: 68

Text #7

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7938. Min Act: -0.1700

Data Index: 8700439 (Open Web Text)

Max Activating Token Index: 665

Text #8

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7530. Min Act: -0.1700

Data Index: 5859749 (Open Web Text)

Max Activating Token Index: 293

Text #9

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7169. Min Act: -0.1700

Data Index: 2289678 (Open Web Text)

Max Activating Token Index: 821

Text #10

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6997. Min Act: -0.1700

Data Index: 6761318 (Open Web Text)

Max Activating Token Index: 453

Text #11

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7268. Min Act: -0.1700

Data Index: 5039896 (Open Web Text)

Max Activating Token Index: 790

Text #12

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6861. Min Act: -0.1700

Data Index: 3289606 (Open Web Text)

Max Activating Token Index: 581

Text #13

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7161. Min Act: -0.1700

Data Index: 1034399 (Open Web Text)

Max Activating Token Index: 44

Text #14

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7088. Min Act: -0.1700

Data Index: 5598996 (Open Web Text)

Max Activating Token Index: 321

Text #15

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6945. Min Act: -0.1700

Data Index: 4078514 (Open Web Text)

Max Activating Token Index: 256

Text #16

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6994. Min Act: -0.1700

Data Index: 6290383 (Open Web Text)

Max Activating Token Index: 744

Text #17

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6989. Min Act: -0.1700

Data Index: 2449050 (Open Web Text)

Max Activating Token Index: 964

Text #18

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.7173. Min Act: -0.1700

Data Index: 2014869 (Open Web Text)

Max Activating Token Index: 478

Text #19

Max Range: 3.9541. Min Range: -3.9541

Max Act: 3.6745. Min Act: -0.1700

Data Index: 4648237 (Open Web Text)

Max Activating Token Index: 759