Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 188 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1912. Min Act: -0.1700

Data Index: 365752 (Open Web Text)

Max Activating Token Index: 445

Text #1

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1770. Min Act: -0.1700

Data Index: 5249427 (Open Web Text)

Max Activating Token Index: 980

Text #2

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1577. Min Act: -0.1700

Data Index: 7687377 (Open Web Text)

Max Activating Token Index: 338

Text #3

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1686. Min Act: -0.1700

Data Index: 8523945 (Open Web Text)

Max Activating Token Index: 140

Text #4

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1480. Min Act: -0.1700

Data Index: 7171542 (Open Web Text)

Max Activating Token Index: 920

Text #5

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1382. Min Act: -0.1700

Data Index: 2782681 (Open Web Text)

Max Activating Token Index: 546

Text #6

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1426. Min Act: -0.1700

Data Index: 2857246 (Open Web Text)

Max Activating Token Index: 342

Text #7

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1463. Min Act: -0.1700

Data Index: 5497956 (Open Web Text)

Max Activating Token Index: 52

Text #8

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1067. Min Act: -0.1700

Data Index: 4064851 (Open Web Text)

Max Activating Token Index: 174

Text #9

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1103. Min Act: -0.1700

Data Index: 3065012 (Open Web Text)

Max Activating Token Index: 889

Text #10

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1062. Min Act: -0.1700

Data Index: 6011997 (Open Web Text)

Max Activating Token Index: 465

Text #11

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0602. Min Act: -0.1700

Data Index: 5925291 (Open Web Text)

Max Activating Token Index: 768

Text #12

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0824. Min Act: -0.1700

Data Index: 3491187 (Open Web Text)

Max Activating Token Index: 313

Text #13

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.1012. Min Act: -0.1700

Data Index: 3088567 (Open Web Text)

Max Activating Token Index: 684

Text #14

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0313. Min Act: -0.1700

Data Index: 917676 (Open Web Text)

Max Activating Token Index: 437

Text #15

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0939. Min Act: -0.1700

Data Index: 6433381 (Open Web Text)

Max Activating Token Index: 299

Text #16

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0676. Min Act: -0.1700

Data Index: 6333227 (Open Web Text)

Max Activating Token Index: 95

Text #17

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0533. Min Act: -0.1700

Data Index: 6050561 (Open Web Text)

Max Activating Token Index: 433

Text #18

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0268. Min Act: -0.1700

Data Index: 1184773 (Open Web Text)

Max Activating Token Index: 897

Text #19

Max Range: 4.1912. Min Range: -4.1912

Max Act: 4.0564. Min Act: -0.1700

Data Index: 3849951 (Open Web Text)

Max Activating Token Index: 113