Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 48 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.7371. Min Act: -0.1700

Data Index: 147080 (Open Web Text)

Max Activating Token Index: 882

Text #1

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.6960. Min Act: -0.1700

Data Index: 8733608 (Open Web Text)

Max Activating Token Index: 767

Text #2

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.4301. Min Act: -0.1700

Data Index: 3151437 (Open Web Text)

Max Activating Token Index: 995

Text #3

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.4513. Min Act: -0.1700

Data Index: 3573907 (Open Web Text)

Max Activating Token Index: 334

Text #4

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3991. Min Act: -0.1700

Data Index: 4729278 (Open Web Text)

Max Activating Token Index: 386

Text #5

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3569. Min Act: -0.1700

Data Index: 5842360 (Open Web Text)

Max Activating Token Index: 790

Text #6

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3598. Min Act: -0.1700

Data Index: 7699822 (Open Web Text)

Max Activating Token Index: 397

Text #7

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3723. Min Act: -0.1700

Data Index: 3251838 (Open Web Text)

Max Activating Token Index: 915

Text #8

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3294. Min Act: -0.1700

Data Index: 3548529 (Open Web Text)

Max Activating Token Index: 994

Text #9

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3244. Min Act: -0.1700

Data Index: 987228 (Open Web Text)

Max Activating Token Index: 678

Text #10

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2516. Min Act: -0.1700

Data Index: 5185081 (Open Web Text)

Max Activating Token Index: 934

Text #11

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3168. Min Act: -0.1700

Data Index: 5092499 (Open Web Text)

Max Activating Token Index: 578

Text #12

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3914. Min Act: -0.1700

Data Index: 7056031 (Open Web Text)

Max Activating Token Index: 632

Text #13

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2953. Min Act: -0.1700

Data Index: 4317973 (Open Web Text)

Max Activating Token Index: 549

Text #14

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2946. Min Act: -0.1700

Data Index: 2888355 (Open Web Text)

Max Activating Token Index: 264

Text #15

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3137. Min Act: -0.1700

Data Index: 6998183 (Open Web Text)

Max Activating Token Index: 283

Text #16

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2758. Min Act: -0.1700

Data Index: 1384511 (Open Web Text)

Max Activating Token Index: 648

Text #17

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2748. Min Act: -0.1700

Data Index: 556226 (Open Web Text)

Max Activating Token Index: 311

Text #18

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.2812. Min Act: -0.1700

Data Index: 1785286 (Open Web Text)

Max Activating Token Index: 633

Text #19

Max Range: 5.7371. Min Range: -5.7371

Max Act: 5.3075. Min Act: -0.1700

Data Index: 4324303 (Open Web Text)

Max Activating Token Index: 883